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INTRODUCTION 

This  project  addresses  the  need  for  research  on  service  delivery  approaches  for  Service 
Members  (SMs)  with  combat-related  physical  or  psychiatric  symptoms,  including  Posttraumatic 
Stress  Disorder  and/or  post-concussive  symptoms.  As  a  primary  care  encounter,  the  post 
deployment  health  reassessment  (PDHRA)  process  is  critical  to  force  health  protection  efforts. 
The  project  will  develop  and  test  the  effectiveness  of  a  targeted  training  and  feedback 
intervention  designed  to  help  providers  increase  SM  reports  of  behavioral  health  concerns  and 
SM  acceptance  of  a  referral  for  further  assessment.  The  project  builds  on  a  previous  evaluation 
of  the  PDHRA  process,  a  collaborative  effort  between  Vanderbilt  University  and  Force  Health 
Protection  and  Readiness,  and  will  be  applicable  to  all  Service  Branches  and  Components.  This 
evaluation  was  contracted  to  Vanderbilt  University  with  the  final  report  available  on  Defense 
Technical  Information  Center  (DTIC)  at  http://handle.dtic.miI/100.2/ADA528063. 

The  project  has  two  aims.  (Aim  1)  Development  of  PDHRA-specific  clinical  guidelines  and 
training  materials  through  collaboration  with  key  national  leaders  and  installation-level 
stakeholders  involved  in  the  PDHRA  process,  and  through  a  secondary  analysis  of  PDHRA  data 
linked  by  provider.  (Aim  2)  Test  of  intervention  effectiveness  at  four  to  six  sites  with  39  primary 
care  providers  who  conduct  PDHRAs.  Providers  will  be  randomly  assigned  to  one  of  two 
interventions  (training  and  ongoing  feedback  or  training  only)  or  to  typical  training  (control 
group).  Outcomes  include  implementation  fidelity  and  quality,  content  analysis  of 
communication  style  from  interview  audiotapes,  secondary  analysis  of  the  PDHRA  form  and  SM 
health  care  utilization,  and  SM  satisfaction  surveys.  Data  will  be  analyzed  using  a  longitudinal 
repeated  measure  slope-as-outcome  model.  A  secondary  analysis  of  PDHRA  data  will  also  be 
conducted  to  identify  risk  factors  in  the  development  of  Posttraumatic  Stress  Disorder.  The 
project  is  a  cooperative  effort  among  Vanderbilt  University,  Force  Health  Protection  and 
Readiness  (FHP&R),  and  Purdue  University.  The  project  period  of  performance  is  30-SEP-09  to 
31-OCT-ll.  This  report  summarizes  Year  One  (30-SEP-09  to  29-SEP-10)  progress  on  scope  of 
work  (SOW)  activities,  key  research  accomplishments,  and  reportable  outcomes.  We  conclude 
by  summarizing  results  to  date  and  projecting  work  to  be  accomplished  through  the  remainder 
of  the  project. 
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BODY  OF  REPORT 

Vanderbilt  University  Scope  of  Work  (SOW)  Tasks 
Task  1.  Timing  of  Approvals  and  IRB  (Yl,  Ml-11) 

Overview 

Task  1  activities  are  oriented  to  ensuring  that  all  proper  approvals  and  IRB  activities  are 
completed  in  a  timely  manner,  so  that  the  provider  intervention  and  other  research  activities 
can  proceed  according  to  schedule.  The  subtasks  for  Year  1  listed  under  Task  1  in  the  SOW  are 
as  follows: 

•  la.  Multi-project  Institutional  Authorization  Agreement  (IAA)  submitted  to  TMA  Exempt 
Determination  Official  to  establish  the  VU  IRB  as  the  IRB  of  record  (Yl,  Ml-2) 

•  lb.  PHDRA  secondary  analysis  protocol  (Year  1,  Ml-4) 

o  Submitted  to  VU  IRB,  estimated  review  time  for  non-human  subjects  protocol 
(Yl,  Ml) 

o  Submitted  to  TMA  Exempt  Determination  Official,  estimated  review  time  (Yl, 
M3-4) 

•  lc.  PDHRA  focus  group  protocol  (Yl,  Ml-4) 

o  Submitted  to  VU  IRB,  estimated  review  time  for  exempt  protocol  (Yl,  Ml-2) 
o  Submitted  to  appropriate  Army  IRBs  and  estimated  review  time  (Yl,  M3-4) 

•  Id.  Training  &  feedback  intervention  study  protocol  (Yl,  M3-10) 

•  Submitted  to  VU  IRB,  estimated  review  time  for  expedited  protocol  (Yl,  M3-4) 

•  Submitted  to  appropriate  Army  IRBs,  estimated  review  time  (Yl,  M5-11).  Note  that 
final  training  materials  will  be  submitted  for  review  in  months  10-11. 

Status 

With  the  exception  of  the  training  and  feedback  intervention  study  protocol,  we  have  obtained 
necessary  approvals  from  the  Army  Medical  Research  and  Materiel  Command  (MRMC)  and 
Vanderbilt  University's  Institutional  Review  Board  (IRB). 

The  Vanderbilt  research  team  submitted  a  non-human  subject  application  for  secondary 
analysis  (Task  lb)  to  Vanderbilt's  IRB  on  12-DEC-09  and  received  approval  on  30-DEC-09.  A 
second  human  subjects  application,  specifying  access  to  provider  and  location  ID  was  submitted 
to  the  VU  IRB  for  expedited  review  on  12-APR-10  and  notification  of  approval  was  received  on 
17-MAY-10.  The  VU  IRB  approval  and  protocol  were  submitted  to  Army  Medical  Research  and 
Materiel  Command  (MRMC)  on  17-MAY-10  and  was  approved  by  MRMC  on  24-MAY-10. 
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Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Task  la  was  not  completed  since  it  was  determined  through  conversations  between  Vanderbilt 
and  MRMC  that  no  IAA  was  necessary  for  this  project. 

The  PDHRA  focus  group  protocol  (Task  lc)  was  not  submitted  due  to  mutual  agreement 
between  FHP&R  and  Vanderbilt  not  to  use  the  focus  group  methodology  at  installations  in  Year 
1  (see  Task  2  for  more  information). 

The  training  and  feedback  intervention  study  protocol  (Task  Id)  has  not  yet  been  submitted 
due  to  circumstances  described  further  in  Task  4. 

Outcomes  and  Next  Steps 

As  detailed  in  Task  4,  the  training  and  feedback  intervention  study  protocol  (Task  Id)  is  in 
development.  We  anticipate  that  it  will  be  submitted  to  Vanderbilt's  IRB  early  in  Year  2.  As  we 
continue  to  review  installations  for  potential  participation  in  the  study,  we  will  collaborate  with 
FHP&R  to  plan  the  PDHRA  provider  focus  groups  (Task  lc)  or  determine  an  appropriate 
alternative  after  receiving  approval  from  the  Army  GOR. 


Task  2  (Aim  1).  PDHRA  Focus  Groups  (Yl,  Ml-2,  5-9) 

Overview 

The  stated  goal  of  Task  2  was  to  conduct  focus  groups  of  key  stakeholders  involved  in  the 
PDHRA  process,  and  to  analyze  the  resultant  data  with  the  intention  of  identifying  key  elements 
for  training  interventions  relevant  to  content,  format,  and  implementation.  The  subtasks  for 
Year  1  listed  under  Task  2  in  the  SOW  are  included  below: 

•  2a.  Recruitment  of  four  to  six  study  sites  (Yl,  Ml-2) 

•  2b.  Development  of  focus  group  protocols  (Yl,  Ml) 

•  2c.  Administration  of  2-hour  focus  groups  conducted  at  each  study  site  (Yl,  M5-7) 

•  2d.  Professional  transcription  of  focus  group  audiotapes  ongoing  as  each  completed  (Yl, 

M5-7) 

•  2e.  Qualitative  analysis  will  be  ongoing  as  each  focus  group  completed  with  aggregation 

of  findings  after  all  completed  (Yl,  M5-9) 

•  2f.  Production  of  preliminary  reports  and  briefings  (Yl,  M8-9). 


Status 

Task  2  was  not  completed  as  proposed  due  to  significant  legislation  that  was  introduced 
relevant  to  this  project  between  the  time  of  the  original  proposal  and  the  start  of  work.  In 
planning  meetings  and  emails  (primarily  in  March,  2010)  FHP&R  and  Vanderbilt  agreed  not  to 
conduct  the  focus  groups  pending  DoD  efforts  related  to  the  legislation.  In  the  interim, 
Vanderbilt  research  team  members  conducted  a  total  of  seven  informal  telephone  interviews 
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with  representatives  from  five  military  installations  between  20-APR-10  and  30-APR-10.  The 
five  installations  were  chosen  on  the  basis  of  previous  contact  during  the  evaluation  of  the 
PDHRA  process  conducted  by  Vanderbilt  prior  to  this  project.  Installation  representatives 
included  medical  officers  and/or  administrators  involved  in  the  PDHRA  process  at  each 
installation.  This  was  a  planning  activity  similar  to  the  gathering  of  Expert  Panel  perspectives 
rather  than  a  research  activity  designed  to  gather  externally  reportable  data. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

H.R.  2647:  National  Defense  Authorization  Act  for  Fiscal  Year  2010  became  Public  Law  111-84 
on  28-OCT-lO.  Section  708  (hereinafter  referred  to  as  NDAA  Sec.  708)  of  the  act  directly 
affected  how  the  PDHRA  process  would  be  conducted  across  Services  and  Components.  In 
brief,  the  legislation  requires  pre-  and  post-deployment  mental  health  assessments  and  training 
for  the  providers  conducting  the  assessments.  As  a  result,  FHP&R  led  efforts  throughout  2010 
to  modify  the  current  mental  health  risk  assessments  relevant  to  deployment  screening, 
develop  the  associated  policy,  and  develop  the  training  content  and  procedures.  On  04-AUG- 
10,  the  NDAA  Sec.  708  policy  document  became  available  and  was  sent  to  Vanderbilt  by  FHP&R, 
although  the  actual  training  content  still  remained  under  development.  On  27-AUG-10,  the 
modified  assessments  and  training  content  became  available  for  public  viewing  in  the  form  of  a 
set  of  PowerPoint  slides  posted  on  FHP&R's  web  site 

(http://fhpr.osd.mil/pdfs/NDAA%20FHP  DHCC.pdf).  As  can  be  seen  in  the  slides,  the  self-report 
portion  of  the  PDHRA  includes  the  same  behavioral  health  questions  as  before,  now  called  the 
Stage  One  screening.  For  SMs  who  meet  scoring  criteria  on  Stage  One  screening,  additional 
detailed  Stage  Two  screenings  will  be  administered.  The  slides  also  present  the  training  content 
for  health  care  providers,  including  information  on  the  screening  tools,  basic  procedures  for 
scoring  and  administration,  suggestions  for  follow-up  questions  in  cases  of  concern,  suggestions 
for  improving  communication  and  interview  style,  and  additional  resources  for  further 
information.  As  of  this  writing,  FHP&R  is  in  the  process  of  developing  a  video  to  accompany  the 
slides  as  part  of  the  training  procedures.  Note  that  Service  Branches  and  Components  are  still  in 
the  process  of  finalizing  their  plans  for  implementing  the  NDAA  Sec.  708  with  few  details 
available  at  this  time  on  how  the  modifications  will  be  incorporated  into  existing  PDHRA 
processes  at  the  site  level.  According  to  FHP&R,  implementation  of  the  modified  assessments 
and  provider  training  is  scheduled  to  begin  in  Spring  2011. 

Outcomes  and  Next  Steps 

Given  that  the  implementation  plans  relevant  to  the  modified  assessments  and  provider 
training  are  still  pending  from  the  Services,  it  is  as  yet  to  be  determined  when  would  be  an 
appropriate  time  to  conduct  the  focus  groups  as  planned.  As  we  continue  to  review 
installations  for  potential  participation  in  the  study,  we  will  collaborate  with  FHP&R  to  plan  the 
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PDHRA  provider  focus  groups  (Task  lc)  or  determine  an  appropriate  alternative  after  receiving 
approval  from  the  Army  GOR. 

The  informal  telephone  interviews  have  substantially  informed  the  intervention  design  process, 
thus  the  information  gathered  during  these  calls  will  be  summarized  here.  The  Vanderbilt  team 
gathered  information  in  seven  major  areas:  Provider  training  and  quality  assurance;  Staff 
Meetings;  Service  Member  Education;  Ideas  on  Potentially  Useful  Feedback  for  Providers; 
Factors  Influencing  Success  or  Failure  of  the  PDFIRA  Process;  and  Willingness  to  Participate  in  a 
Study.  The  goals  associated  with  each  area  are  listed  below,  along  with  brief  summaries  of  the 
telephone  interviews. 

1.  Provider  Training 

Goal:  To  learn  how  providers  at  the  installations  were  trained,  in  order  to  better 
understand  the  pre-intervention  state  of  each  location,  and  also  to  explore  the  extent  to 
which  the  "pre-intervention"  state  was  uniform  across  installations. 

Findings:  Training  of  PDFIRA  providers  was  found  to  be  minimal,  which  was  consistent 
with  our  findings  in  the  previous  evaluation.  At  all  installations  except  for  one,  new 
providers  shadowed  an  experienced  provider  for  one  to  two  days.  Shadowing  occurred 
for  up  to  two  months  at  the  remaining  installation,  but  this  was  due  more  to  delays  in 
getting  necessary  passwords  than  it  was  to  a  planned  training  time  extension.  Most  sites 
also  reported  including  a  one-time  training  on  policies  and  data  entry  (for  example, 
training  with  MEDPROS  or  receipt  of  PowerPoint  slides  that  detail  guidelines  and 
policies  relevant  to  the  PDFIRA).  One  site  conducted  additional  training  via  monthly 
grand  rounds,  and  providers  at  one  other  site  received  supplemental  suicide  prevention 
training  and  training  from  a  psychiatric  nurse. 

2.  Quality  Assurance 

Goal:  To  determine  how  installations  conduct  quality  assurance  activities  (if  any)  in  the 
pre-intervention  state  and  identify  procedures  already  in  place  that  could  integrate  well 
with  the  provision  of  feedback  through  our  study  intervention. 

Findings:  Evaluation  procedures  varied,  including  quarterly  peer  review  and 
performance  review  (one  installation),  monthly  evaluation  (one  installation),  and  annual 
review  with  semi-annual  updates  (one  installation).  Formal  review  did  not  exist  at  the 
remaining  two  installations.  Performance  review  was  generally  not  specific  to  the 
PDFIRA  process,  but  rather  encompassed  multiple  activities  at  the  installation. 

3.  Staff  Meetings 

Goal:  To  learn  how  often  staff  meetings  were  held  and  what  type  of  content  they 
generally  included.  The  study  intervention  will  include  regular  meetings  to  provide 
feedback  to  clinical  and  administrative  staff  associated  with  the  PDFIRA  process,  which 
we  hope  to  integrate  into  existing  installation  processes  so  as  to  cause  a  minimum 
amount  of  disruption  and  increased  time  constraints. 


9 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


Findings:  The  frequency  of  staff  meetings  was  variable.  They  occurred  quarterly  at  one 
installation,  monthly  at  another  (in  the  form  of  grand  rounds),  weekly  at  two 
installations,  and  only  as  needed  at  the  remaining  site.  At  all  installations  except  the  one 
which  employed  a  grand  rounds  format,  typical  topics  included  review  of  procedure 
changes  rather  than  clinical  issues  or  didactic  content  associated  with  effective 
strategies  for  performing  the  PDHRA  interview. 

4.  Service  Member  Education 

Goal:  To  learn  about  any  formal  procedures  for  providing  education  to  Service  Members 
that  existed  at  each  installation,  in  order  to  better  understand  the  pre-intervention  state 
at  these  locations. 

Findings:  Interviewees  from  all  installations  viewed  the  provision  of  education  to  Service 
Members  as  a  role  for  providers,  but  most  locations  filled  this  need  outside  of  the 
interview  with  presentations  like  Battlemind  II  or  post-deployment  briefings.  Brochures 
and  handouts  were  generally  available  for  providers  to  give  to  SMs  during  interviews. 
When  the  interviewee  happened  to  be  a  provider  who  conducted  PDFIRA  screenings,  he 
or  she  tended  to  note  at  least  one  personally  preferred  strategy  for  identifying  Service 
Members  who  could  benefit  from  education  during  the  interview.  For  example,  one 
provider  said  he  always  gave  out  information  to  SMs  on  the  cusp  of  a  problem,  and 
another  said  he  always  made  sure  that  SMs  who  seemed  to  be  trying  to  rush  the 
interview  left  with  information  on  how  to  make  an  appointment  if  they  needed  one. 
However,  no  installation  mentioned  any  formal  training  mechanisms  to  help  providers 
better  communicate  educational  information  to  Service  Members,  and  three  specifically 
stated  that  no  such  education  existed  at  all. 

5.  Ideas  on  Potentially  Useful  Feedback 

Goal:  To  learn  what  forms  of  feedback  would  be  considered  useful  to  key  stakeholders 
at  installations. 

Findings:  Possible  types  of  feedback  for  providers  fell  into  several  categories,  which  are 
detailed  below: 

a.  Deployment  and  Combat  Experiences— All  interviewees  stated  that  information 
about  deployment  conditions  and  number  of  deployments  would  be  useful,  as 
would  more  knowledge  of  actual  experiences  and  injuries.  One  interviewee 
noted  that  the  system  is  concerned  only  with  problems  related  to  the  most 
recent  deployment,  but  that  this  is  not  in  line  with  the  SM's  perspective,  in 
which  all  deployments  run  together,  and  symptoms  related  to  one  deployment 
may  continue  or  reappear  following  subsequent  deployments. 

b.  SM  Satisfaction— Installation  representatives  were  divided  on  the  usefulness  of 
this  potential  category  of  feedback.  One  interviewee  saw  it  as  potentially  very 
important  for  use  in  identifying  and  fixing  problems.  Other  installations  were 
unsure  about  the  extent  to  which  it  would  help,  with  one  site  representative 
noting  that  he  personally  thought  it  could  be  important,  but  was  unsure  of  how 
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interested  providers  would  be  given  that  they  are  more  concerned  with  the 
straightforward  medical  side  of  the  process  rather  than  with  the  communication 
side. 

c.  Feedback  about  Referrals— This  could  include  either  individual  or  aggregate  data 
relating  to  numbers  and  types  of  referrals  as  well  as  information  on  whether 
referrals  were  kept.  In  general,  interviewees  saw  follow-up  of  individual  SMs 
who  had  been  referred  as  a  job  for  someone  other  than  the  PDHRA  provider.  For 
the  most  part,  the  installations  we  contacted  appeared  not  to  have  explored  the 
possibility  of  using  information  on  past  referrals  to  identify  ways  of  improving 
future  appropriate  referral  rates.  Flowever,  one  interviewee  did  note  that  clinic 
providers  sometimes  call  to  report  inappropriate  referrals,  and  the  interviewee 
indicated  that  it  might  be  helpful  to  formalize  this  process  somehow. 

d.  Instruction  on  how  to  use  the  PDFIRA  Form  — More  instruction  on  how  to  best 
use  the  PDFIRA  form  (for  example,  guidance  on  interpreting  self-reported 
information)  was  seen  as  potentially  useful  by  interviewees  from  two  sites, 
though  concern  was  noted  about  the  need  to  do  this  in  such  a  way  that 
providers  were  not  treated  like  robots.  One  interviewee  mentioned  that 
providers  were  very  oriented  towards  sticking  to  the  form,  and  that  a  strategy 
that  would  be  more  useful  than  training  providers  to  ask  more  questions  would 
be  to  simply  add  these  questions  to  the  form. 


6.  Factors  Influencing  Success  or  Failure  of  the  PDHRA  Process 

Goal:  To  determine  factors  that  key  personnel  at  each  installation  considered  to  be 
essential  for  the  success  of  the  PDHRA  process. 

Findings:  All  interviewees  had  some  specific  ideas  on  factors  that  could  influence  the 
effectiveness  of  the  PDHRA  process.  A  summary  of  these  considerations  is  listed  below: 

a.  The  site's  ability  to  facilitate  referrals  was  listed  as  key  by  two  sites.  One 
commonly  cited  factor  in  referral  facilitation  was  the  practice  of  having  psych 
techs  or  other  mental  health  professionals  available  so  that  the  referral  could 
take  place  immediately  once  it  had  been  made.  All  but  one  site  already  had  this 
latter  structure  in  place.  This  ability  to  facilitate  referrals  was  also  seen  as 
demonstrating  to  SMs  that  they  would  get  the  care  they  needed. 

b.  Two  sites  emphasized  the  importance  of  having  providers  who  were  retired 
military  or  at  least  knowledgeable  about  the  military.  This  background  was  seen 
as  helpful  in  recognizing  problems  and  in  knowing  what  to  expect  during  the 
interview. 

c.  One  site  noted  that  it  would  be  helpful  if  the  PDHRA  were  moved  to  a  more 
clinical  setting  so  that  SMs  would  spend  less  time  waiting  in  line,  and  there 
would  be  less  combination  of  the  PDHRA  with  vaccinations  and  other  medical 
readiness  procedures. 

d.  One  interviewee  stated  a  belief  that  more  automation  of  the  system  would 
positively  influence  success.  She  thought  that  having  SMs  complete  a  more 
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extensive  online  screening  with  validated  instruments  would  provide  a  better 
(direct)  conduit  to  treatment  than  using  the  current  PDHRA  form  and  interview 
for  screening.  This  interviewee  also  thought  that,  failing  this,  it  was  important  to 
get  all  of  the  relevant  symptoms  and  information  to  the  providers  ahead  of  time, 
e.  One  site  already  had  a  process  in  place  by  which  it  received  information  about 
deployment  and  specific  medical  problems  from  the  unit  medic  before  the  unit 
returned.  This  was  seen  as  a  factor  influencing  success  of  the  process. 

7.  Willingness  to  participate  in  Study 

Goal:  To  assess  willingness  to  participate  in  an  intervention  during  Year  2  of  the  project 
and  to  determine  whether  representatives  were  also  willing  to  be  contacted  to  give 
further  input  during  intervention  development. 

Findings:  All  interviewees  were  willing  to  be  contacted  with  further  questions  in  the 
future,  and  all  expressed  potential  interest  in  participating  in  the  proposed  study. 


Task  3  (Aim  1).  PDHRA  Secondary  Analysis  (Yl,  M4-9) 

Overview 

The  stated  goal  of  Task  3  was  to  conduct  an  extensive,  robust  secondary  analysis  of  the  PDHRA 
data  obtained  during  Vanderbilt's  previous  DoD-funded  evaluation  of  the  PDHRA  process,  with 
a  focus  on  identifying  provider  factors  that  contribute  to  elicitation  of  candid  SM  reporting  of 
behavioral  health  concerns  and  to  SM  acceptance  of  associated  referrals  for  further  evaluation 
or  treatment.  The  resulting  information  would  be  utilized  in  the  development  of  the  training 
and  feedback  intervention.  The  subtasks  for  Year  1  listed  under  Task  3  in  the  SOW  are  included 
below: 


•  3a.  Data  requests  to  appropriate  information  technology  officer  at  each  Service  for 
provider  and  MTF  identifiers  for  PDHRAs  completed  between  01-JAN-06  to  31-MAY-09 
(Yl,  M4) 

•  3b.  Linking  file  created  by  TMA  to  provide  de-identified  dataset  to  VU  containing  non¬ 
identifying  SM  identifier  and  provider/MTF  identifiers  (Yl,  M4-8) 

•  3c.  Data  management  and  analysis  (Yl,  M5-7).  Abbreviated  analytic  timeframe 
estimated  because  we  will  be  adding  this  dataset  to  existing  clean  datasets  with  much  of 
the  analytic  programming  developed. 

•  3d.  Production  of  preliminary  reports  and  briefings  (Yl,  M8-9). 
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Status 

Data  Use  Agreements  (DUAs)  for  Vanderbilt  to  receive  provider  and  MTF  identifiers  for  PDHRAs 
completed  between  01-JAN-06  and  31-MAY-09  were  signed  and  in  place  with  all  Services  ahead 
of  schedule  (Task  3a).  The  Air  Force  DUA  was  signed  by  all  parties  on  4-MAR-10,  the  Army  DUA 
on  7-APR-10,  and  the  Navy  DUA  on  20-APR-10. 

To  date  no  data  have  been  received  by  Vanderbilt;  thus  tasks  3b,  3c,  and  3d  remain  incomplete. 
Vanderbilt  took  the  opportunity  to  conduct  additional  psychometric  analyses  on  SM  survey 
data  originally  collected  as  part  of  the  previous  PDFIRA  evaluation.  The  goal  of  these  analyses 
was  to  further  refine  the  SM  survey  for  use  during  the  current  study. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

We  clarified  that  data  were  to  be  received  from  Armed  Forces  Health  Surveillance  Center 
(AFHSC)  and  not  TMA.  A  DUA  was  signed  by  all  parties  on  14-JAN-10;  however,  this  DUA  was 
later  rescinded  by  AFHSC  on  29-APR-10  because  it  did  not  contain  language  giving  AFHSC 
authority  to  edit  and  approve  any  presentations  or  publications  resulting  from  the  study.  In 
addition  the  original  DUA  was  not  valid  as  it  was  not  signed  by  the  appropriate  approval 
authority  at  Vanderbilt.  A  new  AFHSC  DUA  was  provided  to  Vanderbilt  on  21-JUN-10.  This  DUA 
contained  language  giving  AFHSC  authority  to  edit  and  approve  any  presentations  or 
publications  resulting  from  the  study.  This  language  was  found  to  be  unacceptably  restrictive  by 
Vanderbilt's  Division  of  Sponsored  Research  (DSR),  the  official  signing  body  for  Vanderbilt.  After 
several  weeks  of  negotiation  among  FHP&R,  AFHSC,  the  Vanderbilt  research  team,  and  the 
Vanderbilt  DSR  it  was  determined  that  the  only  way  to  achieve  resolution  of  the  AFHSC  DUA 
issue  was  for  the  Vanderbilt  research  team  to  request  and  receive  a  Waiver  of  University 
Information  Dissemination  Policy  from  the  Vice  Provost  of  Vanderbilt  University.  It  must  be 
emphasized  that  this  was  an  extreme  measure,  and  that  such  waivers  are  only  very  rarely 
granted  by  the  University.  In  this  case,  the  waiver  was  approved  on  29-SEP-10.  It  is  expected 
that  the  DUA  will  be  signed  early  in  Year  2  and  data  received  within  six  weeks  of  provision  of 
the  signed  DUA  to  AFHSC.  Data  management,  analysis,  and  production  of  preliminary  reports 
and  briefings  will  subsequently  follow  in  Year  2. 

Outcomes  and  Next  Steps 

Vanderbilt  currently  awaits  the  data  for  secondary  analyses,  and  will  carry  out  the  activities 
originally  scheduled  to  take  place  during  Year  1  upon  their  receipt. 

The  results  of  the  psychometric  analyses  using  the  existing  data  from  the  SM  survey  are 
presented  in  Appendix  A.  The  survey  had  been  previously  administered  to  6,714  SMs  to  gather 
information  on  SM  satisfaction  with  and  attitudes  relevant  to  the  PDHRA  process.  Nine  scales 


13 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


had  been  formed  on  the  basis  of  theory  review  and  found  to  be  reliable.  The  data  were  re¬ 
analyzed  using  exploratory  factor  analysis,  resulting  in  seven  scales.  As  the  training  and 
feedback  intervention  and  accompanying  research  design  are  finalized  early  in  Year  2,  the  SM 
survey  with  new  scales  will  be  finalized.  It  is  likely  that  additional  items  may  be  included  after 
the  design  is  finalized. 

Task  4  (Aim  2).  Training  and  Feedback  Intervention  Effectiveness  Study  (Yl,  Ml- 

9) 

Overview 

The  activities  listed  under  Task  4  address  the  central  goal  of  Vanderbilt's  research,  which  is  to 
develop  and  test  the  effectiveness  of  a  targeted  training  and  feedback  intervention  designed  to 
help  providers  increase  Service  Member  reports  of  behavioral  health  concerns  and  Service 
Member  acceptance  of  referrals  for  further  assessment.  The  subtasks  for  Year  1  listed  under 
Task  4  in  the  SOW  are  as  follows: 

•  4a.  Recruitment  of  four  to  six  study  sites  (Yl,  Ml-2) 

•  4b.  Development  of  training  materials  (Yr  1,  Ml-9) 

•  4c.  Randomization  of  39  providers  across  four  to  six  study  sites  (Yl,  M12) 

•  4d.  Collection  of  pre-training  audiotapes  from  39  providers,  consisting  of  one  randomly 
selected  hour  of  PDHRA  interviews  (Yl,  M12) 


Status 

As  stated  previously  in  Task  2,  the  five  installations  contacted  in  the  course  of  gathering 
information  from  key  stakeholders  in  the  PDHRA  process  all  expressed  interest  in  potentially 
participating  in  the  study.  Because  the  training  and  feedback  intervention  must  occur  at 
installations  with  sufficient  throughput  of  SMs  participating  in  PDHRA  screening,  FHP&R 
contacted  the  Army  for  deployment  schedule  information  through  March  of  2011  (the  PDHRA  is 
typically  scheduled  for  three  to  six  months  post-deployment).  This  information  was  received  by 
FHP&R  in  September  2010.  FHP&R  communicated  to  Vanderbilt  that  the  five  potential 
installations  would  have  sufficient  throughput  through  March  2011.  At  this  time,  no  study  sites 
have  been  formally  recruited.  Thus  no  randomization  of  providers  or  data  collection  has 
occurred. 

The  development  of  the  provider  training  and  feedback  intervention  is  in  progress  but  not  yet 
complete  due  to  delays  described  further  below. 
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Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Three  areas  of  concern  contributed  to  delays  in  the  study  design:  (1)  lack  of  completion  of  an 
agreed  upon  DUA  to  obtain  the  data  for  secondary  analysis;  (2)  the  2010  NDAA  Sec.  708  levied 
additional  requirements  that  were  not  anticipated  at  the  outset  of  the  study,  which  directly 
impacted  the  study  design;  and  (3)  logistic  concerns  on  the  feasibility  of  a  data-based  feedback 
and  decision-support  system  based  on  use  of  PDHRA  as  the  data  source.  These  three  issues  and 
Vanderbilt's  related  progress  in  intervention  design  are  summarized  below. 

First,  the  primary  activity  during  Year  1  was  intended  to  be  an  extensive  secondary  analysis  of 
the  data  collected  during  Vanderbilt's  previous  evaluation  of  the  PDHRA  process  (see  Task  3). 

By  adding  variables  to  account  for  clustering  of  SMs  within  providers  within  installations,  this 
analysis  would  inform  the  development  of  an  evidence-based  training  and  feedback 
intervention  by  identifying  areas  of  variability  in  concerns  and  referrals  attributed  to  the 
provider,  over  and  above  SM  self-reported  problems.  The  secondary  analysis  by  Vanderbilt  in 
addition  to  new  VA  data  analyzed  by  Purdue  was  also  intended  to  form  the  basis  of  the 
actuarial  modeling  approach  described  in  the  original  proposal  that  was  awarded.  Since  no  data 
have  yet  been  received,  Vanderbilt  conducted  a  review  of  the  literature  on  relevant  strategies 
to  enhance  provider  communication  and  decision  making  (included  as  Appendix  B).  This 
literature  review  summarized  studies  that  have  evaluated  the  effectiveness  of  several  types  of 
interventions  to  improve  provider  behavior,  including  training  on  communication  strategies  to 
enhance  elicitation  of  patient  problems  and  compliance  with  medical  regimen;  feedback  on 
clinically  relevant  and  actionable  information  related  to  the  PDHRA  process;  and  quality 
indicators  of  strategies  for  decision  support  to  health  care  providers.  The  section  on  feedback 
focused  on  studies  of  published  validity  and  utilization  of  the  behavioral  health  subscales 
contained  in  the  PDHRA  for  identifying  behavioral  problems  in  screening  populations. 

In  addition,  Vanderbilt  focused  efforts  on  reviewing  our  previous  evaluation  report  for 
information  relevant  to  enhancing  PDHRA  provider  reliability  (i.e.,  decreasing  variability  in 
provider  reporting  of  concerns  and  referral  recommendations  for  SMs  with  similar  problems) 
and  sensitivity  (i.e.,  increasing  elicitation  of  SM  concerns  and  problems  when  present).  The 
previous  evaluation  findings  were  presented  to  FHP&R  staff  and  others  at  an  in-person  meeting 
on  14-JAN-10.  A  second  teleconference  meeting  was  held  on  03-MAR-10  with  staff  from  FHP&R 
and  Defense  Centers  of  Excellence  for  Psychological  Health  and  Traumatic  Brain  Injury  (DCoE) 
for  the  purpose  of  communication  information  relevant  to  intervention  design.  The  slide 
presentation  from  03-MAR-10  meeting  is  included  as  Appendix  C. 

Second,  the  NDAA  Sec.  708  legislation  previously  described  (see  Task  2)  significantly  influenced 
our  progress  in  designing  the  training  and  feedback  intervention.  Until  August  2010, 
comprehensive  details  of  the  policy  and  associated  materials  were  unavailable  to  us,  as  they 
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were  in  development.  This  contributed  to  concerns  that  any  training  activities  developed  as 
part  of  this  project  could  potentially  be  redundant  or  even  contradictory  to  the  NDAA  Sec.  708 
policy,  which  would  negatively  impact  not  only  the  pilot  but  also  utilization  of  any  findings  for 
future  implementation  after  project  completion.  Therefore,  in  March  2010,  FHP&R  and 
Vanderbilt  agreed  to  focus  Year  1  efforts  on  the  feedback  aspects  of  the  intervention. 
Vanderbilt  developed  an  initial  intervention  design  focusing  on  two  types  of  information  that 
could  enhance  provider  behavior:  decision-support  incorporating  actuarial  modeling 
approaches  that  could  be  used  by  providers  with  individual  SMs,  and  aggregated  weekly 
feedback  on  primary  PDHRA  outcomes  that  could  be  used  for  quality  improvement  purposes. 
The  design  was  presented  to  FHP&R  and  approved  prior  to  a  presentation  to  the  Expert  Panel 
on  21-JUN-10.  A  description  of  the  preliminary  intervention  design  including  examples  is 
available  in  the  presentation  slides  in  Appendix  D.  Examples  contained  in  the  slides  provide 
mock-ups  using  a  computer-based  system  and  pen-and-paper  protocols. 

Third,  the  initial  intervention  design  was  predicated  on  the  use  of  PDHRA  data  as  the  source  for 
the  data-based  feedback  and  decision-support  system.  This  would  require  frequent  downloads 
of  PDHRA  data  and  a  complex  de-identification/re-identification  procedure  (for  use  by 
Vanderbilt  in  creating  the  feedback  and  decision-support  for  providers).  Although  the  feasibility 
of  using  PDHRA  data  was  yet  to  be  determined,  with  concerns  raised  by  FHP&R  in  March  2010, 
both  Vanderbilt  and  FHP&R  agreed  to  explore  the  possibility  with  intervention  development 
ensuing  over  the  next  several  months.  In  early  September  2010  FHP&R  asked  Vanderbilt  to 
reconsider  the  intervention  design,  which  was  considered  too  resource  intensive  because  of 
information  technology  (IT)  limitations,  including  the  need  for  weekly  involvement  of  AFHSC  for 
downloading  and  de-/re-identifying  PDHRA  data.  In  addition,  further  discussions  with  FHP&R 
clarified  feasibility  concerns  about  the  inclusion  of  the  SM-specific  decision-support  and  use  of 
IT  for  delivery  due  to  timeline  and  resource  limitations.  Also  in  September  2010  the  policy  and 
training  materials  for  NDAA  Sec.  708  had  been  made  publicly  available.  FHP&R  asked  that 
Vanderbilt  focus  efforts  on  training  related  to  provider  communication  and  interview  style  as 
stated  in  the  original  proposal,  stating  this  would  be  a  valuable  addition  to  the  new  NDAA  Sec. 
708  materials.  At  this  time,  Vanderbilt  is  in  the  process  of  revising  the  training  and  feedback 
intervention  design.  In  addition  to  internal  planning  efforts,  the  Vanderbilt  team  has  a 
scheduled  in-person  meeting  with  Sarah  Mustillo  from  Purdue  University  for  a  day-long 
planning  meeting. 

Outcomes  and  Next  Steps 

It  is  anticipated  that  the  research  team  will  collaborate  with  FHP&R  to  develop  the  final  plan 
early  in  Year  2  to  be  followed  by  the  necessary  approvals  processes. 
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Given  the  delays  experienced,  it  is  anticipated  that  the  intervention  will  be  implemented  in 
May,  2011  with  data  collection  continuing  for  a  period  of  four  months.  FHP&R  will  determine 
appropriateness  of  potential  participating  installations  on  the  basis  of  throughput  of  SMs 
participating  in  PDHRA  screenings  through  the  period  of  May  to  August,  2011.  FHP&R  plans  to 
formally  request  participation  from  the  Army  for  formal  site  recruitment  after  the  approvals 
processes  are  in  place.  Randomization  of  providers  and  pre-intervention  data  collection  will 
occur  prior  to  implementation  of  the  intervention  anticipated  in  May,  2011.  As  possible,  pre¬ 
intervention  data  collection  will  occur  prior  to  the  implementation  of  the  NDAA  Sec.  708 
activities  at  participating  installations. 

Task  5.  Expert  Panel  Meetings  (Yl,  M2,  9) 

Overview 

The  purpose  of  the  Expert  Panel  meetings  is  to  ensure  that  intervention  development  is  fully 
informed  by  the  needs  and  resources  of  all  Service  Branches  and  Components.  The  subtasks  for 
Year  1  listed  under  Task  6  in  the  SOW  are  as  follows: 

•  5a.  Four-hour  in-person  meeting  in  Washington  DC  (Yl,  Ml) 

•  5b.  Two-hour  teleconference  calls  (Yl,  M9). 

Status 

The  Expert  Panel  was  formed  by  FHP&R  with  suggestions  for  potential  members  provided  by 
Vanderbilt.  FHP&R  contacted  potential  participants  in  the  month  of  June  resulting  in  a  final 
panel  of  eight  members  by  23-JUN-10.  A  ninth  member  was  added  to  the  panel  in  August 
resulting  in  a  final  group  of  nine  members  representing  expertise  and  leadership  in  areas 
relevant  to  the  PDHRA  process.  The  membership  roster  is  included  as  Appendix  E. 

The  first  in-person  Expert  Panel  meeting  was  held  on  21-JUL-10.  The  meeting  was  scheduled  for 
two  hours,  with  an  initial  presentation  of  the  training  and  feedback  intervention  design  by 
Vanderbilt  (see  Appendix  D  previously  referenced  in  Task  4)  followed  by  open  discussion.  The 
goal  of  the  meeting  was  to  present  options  for  intervention  design  and  elicit  members' 
thoughts  and  opinions  to  guide  development.  One  member  attended  by  telephone.  Members 
who  could  not  attend  the  meeting  were  provided  with  the  presentation  slides  by  email. 

Two  follow-up  meetings  were  scheduled  with  one  of  the  Expert  Panel  members,  COL  Engel  who 
has  developed  a  computerized  information  system  called  First  Steps  in  support  of  the  RESPECT- 
Mil  program.  RESPECT-Mil  is  a  primary  care  model  employed  in  select  Army  installations  to 
improve  the  identification  and  management  of  behavioral  health  conditions.  The  goal  of  these 
meetings  was  to  learn  more  about  COL  Engel's  existing  computerized  information  system  and 
to  further  discuss  the  potential  use  of  computerized  decision  support  as  part  of  the  study 
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intervention  for  provision  of  feedback  to  providers.  The  two  one-hour  teleconference  calls 
occurred  on  26-AUG-10  (attended  by  Vanderbilt  team,  FHP&R,  and  COL  Engel's  group)  and  17- 
SEP-10  (attended  by  Vanderbilt  team  and  COL  Engel's  group). 

A  ninety  minute  teleconference  call  among  FHP&R,  the  full  Expert  Panel,  and  Vanderbilt  had 
initially  been  scheduled  for  08-OCT-10,  but  was  cancelled  on  28-SEP-10  due  to  a  joint  FHP&R- 
Vanderbilt  decision  to  conduct  further  work  on  the  intervention  design  before  asking  the  Expert 
Panel  to  devote  time  to  the  development  process. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

As  described  previously,  the  development  of  the  training  and  feedback  intervention  design  was 
delayed  due  to  the  need  to  better  understand  the  implications  of  the  NDAA  Sec.  708  legislation. 
Thus,  the  formation  of  the  Expert  Panel  did  not  commence  until  after  an  initial  design  had  been 
developed  by  Vanderbilt  and  approved  by  FHP&R  for  presentation  to  the  panel.  Further 
modifications  to  the  intervention  and  accompanying  research  design  necessitated  cancelling 
the  08-OCT-10  meeting  (see  Task  4  for  more  detail). 

In  addition,  due  to  the  busy  schedules  of  the  Expert  Panel  members,  it  was  determined  that 
shorter  meetings  (two  instead  of  four  hours)  with  break-out  meetings  scheduled  as  needed 
would  improve  participation. 

Outcomes  and  Next  Steps 

The  Expert  Panel's  response  to  the  initial  training  and  feedback  intervention  design  was 
favorable,  with  strong  agreement  for  the  need  to  improve  reliability  of  provider 
recommendations  for  referral  resulting  from  the  PDHRA.  After  Vanderbilt  and  FHP&R  have 
completed  the  revisions  to  the  intervention  design,  the  next  Expert  Panel  teleconference  will  be 
scheduled.  It  is  anticipated  to  occur  early  in  Year  2. 


Task  6.  Project  Planning  Meetings  (Yl,  All  months) 

Overview 

The  planning  meetings  outlined  in  Task  6  are  to  ensure  that  both  the  development  of  the 
intervention  and  the  resolution  of  any  problems  that  might  arise  could  be  dealt  with  in  a 
collaborative  fashion  by  Vanderbilt,  FHP&R,  and  Purdue  (Vanderbilt's  subcontractor).  The 
subtasks  for  Year  1  listed  under  Task  6  in  the  SOW  are  as  follows: 

•  6a.  Weekly  one-hour  teleconference  calls  (Yl,  all  months). 
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•  6b.  Three  one-day  intensive  project  meetings  to  be  held  at  FHP&R  in  Washington,  DC 

(Yl,  Ml,  5,  10). 


Status 

After  FHP&R  assigned  the  current  project  manager,  LCDR  Nicole  Frazer,  the  weekly  one-hour 
teleconference  calls  commenced  in  January,  2010.  Calls  were  productive  and  generally 
completed  as  scheduled.  MAJ  Bonilla-Vasquez,  the  Grant  Officer's  Representative,  attends 
meetings  in  addition  to  FHP&R,  Purdue,  and  Vanderbilt  staff.  Additional  teleconferences  were 
scheduled  as  needed,  with  frequent  email  communication.  A  table  of  all  external  meetings 
(project  planning  meetings.  Expert  Panel  meetings,  and  other  assorted  meetings)  is  included  as 
Appendix  F.  In  addition  to  these  meetings  the  Vanderbilt  research  team  meets  internally  at 
least  once  each  week. 

Two  in-person  intensive  project  meetings  were  held  at  FHP&R  on  14-JAN-10  and  21-JUL-10.  The 
first  meeting  served  to  introduce  the  team  members  to  each  other  and  begin  mutual  review  of 
the  previous  evaluation  findings  relevant  to  the  study  development.  The  second  meeting  was  a 
follow-up  discussion  after  the  Expert  Panel  meeting.  An  in-person  meeting  between  the 
Vanderbilt  research  team  and  Sarah  Mustillo  of  Purdue  University  (subcontract)  has  been 
scheduled  for  30-SEP-10  at  Vanderbilt.  The  goal  of  this  meeting  is  to  make  revisions  to  the 
intervention  and  research  design  based  on  feedback  from  FHP&R  that  necessitated  further 
modifications  (see  Task  4  for  further  detail). 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Vanderbilt  was  not  assigned  a  permanent  project  manager  until  January,  2010,  which  delayed 
the  start  of  project  planning  meetings. 

Outcomes  and  Next  Steps 

The  weekly  teleconferences  have  allowed  Vanderbilt  to  receive  frequent  updates  regarding 
important  military  and  government  factors  influencing  the  design  of  the  intervention,  and  have 
also  provided  a  venue  for  ongoing  collaboration.  It  is  anticipated  that  project  planning  meetings 
will  continue  to  be  held  as  scheduled  during  Year  2. 
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Purdue  University  Scope  of  Work  (SOW)  Tasks 

Task  1.  Dataset  creation  and  preparation  at  VHA  and  DoD  (Yl,  Ml-4) 

Overview 

Personnel  at  FHP&R  and  VHA  will  collect  existing  data  on  target  population.  Purdue  personnel 
will  establish  data  collection  procedures,  coordinate  data  collection  between  those  two  sites, 
troubleshoot,  and  assist  with  technical  consultation. 

Status 

We  have  obtained  necessary  approvals  from  MRMC,  VHA,  and  VA.  At  the  VA,  we  have  applied 
for  and  received  approval  from  the  local  R&D  committee,  the  privacy  board,  and  the  local 
institution.  At  the  National  VHA  level,  we  have  submitted  all  necessary  documents  and  Dr. 
Mayeda  at  the  VA  has  been  granted  approval  for  the  identifiable  data.  We  have  hired  a  data 
manager  to  complete  the  data  download  and  create  the  dataset. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Because  the  approvals  processes  took  substantially  longer  than  expected  on  both  the  DoD  and 
VHA  ends,  we  have  yet  to  coordinate  collection  of  existing  data  between  the  two  sites,  as  no 
data  have  been  obtained  yet  from  either  the  DoD  or  the  VA. 

Outcomes  and  Next  Steps 

Now  that  the  approvals  processes  have  been  cleared,  we  will  work  with  Vanderbilt  for  the 
FHP&R  data  and  with  the  VA  to  obtain  the  VHA  data.  We  will  also  work  closely  with  FHP&R  to 
coordinate  the  receipt  of  the  linking  file  by  the  VA,  so  that  they  may  assign  the  unique  study 
identifier  and  remove  actual  ID  numbers  before  sending  to  Purdue. 


Task  2.  Travel  to  DC  and  Indianapolis  to  consult  on  collection  of  existing 
data  (Yl,  M3-5) 

Overview 

Purdue  personnel  will  travel  as  necessary  to  finalize  what  data  are  being  obtained  from  existing 
records  as  well  as  help  establish  procedures  for  data  management. 


Status 

Purdue  personnel  have  participated  in  weekly  phone  calls  with  DoD  and  Vanderbilt  personnel 
and  have  met  by  phone  as  needed  with  VA  staff  in  Indianapolis. 
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Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Because  the  approvals  were  not  yet  in  place  at  either  the  DoD  or  the  VA/VHA,  no  physical  travel 
was  necessary. 

Outcomes  and  Next  Steps 

Now  that  the  approvals  processes  have  been  cleared,  we  will  arrange  an  in-person  meeting  in 
Indianapolis  to  finalize  what  variables  are  being  obtained  from  the  medical  records  data. 

Task  3.  Obtain  data  from  FHP&R  and  VA  and  create  dataset  (Yl,  M4-8) 

Overview 

Purdue  personnel  will  obtain  the  data  from  both  the  VA  and  the  DoD  and  create  a  dataset  for 
the  project.  They  will  clean,  recode,  merge,  and  reshape  data  as  necessary  to  prepare  for 
analyses. 

Status 

No  dataset  preparations  have  occurred,  but  Mustillo  did  participate  in  a  workshop  on 
downloading  and  managing  the  VHA  data  given  by  VHA  staff.  Mustillo  has  had  several  phone 
meetings  with  VHA  staff  to  discuss  data  and  data  management. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Because  the  data  have  yet  to  be  received  by  Purdue,  we  were  unable  to  complete  any  data 
management  tasks.  Instead,  Purdue  completed  a  literature  review  about  PTSD  in  military 
Service  Members  (Appendix  G)  to  determine  the  current  state  of  knowledge  and  pressing 
issues.  We  focused  the  literature  review  on  two  major  issues:  a)  reviewing  studies  estimating 
incidence  and  prevalence  to  determine  what  accounts  for  differences  in  rates  across  studies 
(e.g.,  samples,  definitions,  screening  tools,  timeframe,  etc.)  and  b)  reviewing  studies  that 
examine  Service-level,  family/community-level,  and  individual-level  risk  and  protective  factors 
for  PTSD.  Each  of  these  areas  will  help  us  frame  our  analyses  and  meet  the  specific  aims  of  the 
project. 

Outcomes  and  Next  Steps 

Now  that  the  approvals  processes  have  been  cleared,  Purdue  expects  to  receive  the  data  in  the 
early  part  of  Year  2  and  will  thus  be  able  to  complete  this  step  at  that  time. 

Task  4.  Analyze  data  (Yl,  M8-12) 

Overview 

Purdue  personnel  will  analyze  the  merged  data  from  FHP&R  and  VA/VHA  according  to  the 
specific  aims  and  the  research  questions  outlined  in  the  proposal. 
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Status 

Because  the  data  have  not  been  received  yet,  no  data  analyses  have  been  conducted. 

Problems  and  Circumstances  that  Necessitated  Changes  to  Task 

Because  the  data  have  yet  to  be  received  by  Purdue,  we  were  unable  to  complete  any  data 

analyses. 

Outcomes  and  Next  Steps 

Now  that  the  approvals  processes  have  been  cleared,  Purdue  expects  to  receive  the  data  in  the 

early  part  of  Year  2  and  will  thus  be  able  to  complete  this  step. 

KEY  RESEARCH  ACCOMPLISHMENTS 

•  Approvals  and  IRB  processes  were  completed  for  both  Vanderbilt  and  Purdue,  with  the 
exception  of  the  approvals  for  the  intervention  study  itself  (submission  planned  early  in 
Year  2)  and  the  focus  groups  (to  be  determined) 

•  Staff  and  protocols  are  in  place  at  the  VA  for  Purdue  to  begin  receiving  VA  data. 

•  In  lieu  of  focus  groups,  informal  telephone  interviews  were  held  with  key  stakeholders 
involved  with  the  PDHRA  process  at  five  military  installations  to  further  understand 
existing  training  and  quality  assurance  procedures  relevant  to  the  design  of  the 
intervention.  Focus  groups  will  be  scheduled  during  Year  2  or  modified  (with  approval 
by  Army  GOR). 

•  The  SM  survey  developed  for  the  previous  evaluation  has  been  updated  based  on 
further  psychometric  analysis,  including  exploratory  factor  analysis  and  Cronbach's 
alpha. 

•  Relevant  literature  review  has  been  conducted  by  both  Vanderbilt  and  Purdue,  which 
will  improve  our  ability  to  meet  the  stated  project  aims. 

•  Significant  progress  has  been  made  on  the  training  and  feedback  intervention  design 
despite  delays,  with  a  final  design  anticipated  in  early  Year  2. 

•  The  Expert  Panel  has  been  convened  with  nine  members  representing  important  areas 
of  leadership  and  expertise  relevant  to  the  study.  One  meeting  was  held  with  the  whole 
panel  and  additional  break-out  meetings  occurred  with  individual  members.  Expert 
Panel  meetings  will  continue  to  be  held  in  Year  2  to  provide  guidance  and  logistic 
support  to  the  project  as  it  continues. 

•  With  the  exception  of  the  fourth  quarter  of  2009,  project  planning  meetings  were  held 
as  scheduled  after  the  permanent  project  manager  was  assigned  by  FHP&R.  Meetings 
will  continue  to  be  held  as  scheduled  during  Year  2 
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REPORTABLE  OUTCOMES 

To  date  one  conference  presentation  has  been  conducted,  with  additional  manuscripts  for 
presentation  and  publication  being  planned.  The  presentation  slides  are  included  as  Appendix 
H.  The  presentation  informed  the  audience  about  the  findings  of  the  previous  evaluation 
conducted  by  Vanderbilt,  highlighting  the  recommendations  for  guidance  to  PDHRA  providers 
to  improve  screening  of  SM  behavioral  health  concerns  and  problems.  Attendance  at  the 
conference  provided  the  opportunity  to  receive  feedback  on  findings  from  key  stakeholders 
involved  in  the  PDHRA  process  relevant  to  the  current  study  and  have  informal  conversations 
with  installations  about  potential  interest  in  participating  in  the  current  study. 

Kelley,  S.D.,  and  Bickman,  L.  (February,  2010).  Evaluation  of  the  Post-Deployment  Health 
Reassessment  (PDHRA)  Process.  Invited  presentation  at  the  Army  PDHRA  conference  in  Falls 
Church,  Virginia. 

CONCLUSION 

Progress  in  completing  planned  Year  1  SOW  activities  has  been  delayed  due  to  several  factors, 
including:  (1)  delays  in  completing  an  agreed  upon  data  use  agreement  for  data  required  by 
Vanderbilt  and  Purdue  to  complete  analyses  relevant  to  the  training  and  feedback  intervention 
design;  and  (2)  the  2010  NDAA  Sec.  708  levied  additional  requirements  that  were  not 
anticipated  at  the  outset  of  the  study,  which  directly  impacted  the  study  design.  Despite  these 
delays,  substantial  work  has  been  completed  in  literature  review,  in  psychometric  analysis  of 
materials  to  be  used  in  the  study,  in  staffing  and  approval  processes  completed,  in  completion 
of  data  use  agreements  with  the  Army,  Navy,  and  Air  Force,  in  formation  and  receipt  of 
guidance  by  the  Expert  Panel,  and  in  development  of  a  strong  working  relationship  among 
FHP&R,  Vanderbilt,  and  Purdue. 

Year  2  of  the  period  of  performance  is  expected  to  be  highly  productive.  With  the  DUA  signed, 
it  is  expected  that  data  will  be  received  early  in  Year  2.  The  training  and  feedback  intervention 
design  is  currently  under  revision  with  anticipated  finalization  of  the  design  in  early  Year  2. 
Implementation  of  the  pilot  is  expected  by  May,  2011  with  data  collection  to  continue  for  four 
months.  This  will  be  followed  by  analysis  and  manuscript  preparation.  Vanderbilt  expects  these 
activities  to  extend  beyond  the  current  period  of  performance  and  plans  to  request  a  one-year 
extension  to  allow  adequate  time  to  complete  the  SOW. 
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Introduction 

In  populations  at  high  risk  for  mental  health  problems,  it  is  beneficial  to  screen  for  such 
problems  so  that  affected  individuals  can  begin  treatment  early  and  potentially  avoid  long  term 
complications  or  problems.  Screening  can  be  an  effective,  low  cost  way  to  evaluate  populations 
at  high  risk  for  health  problems  and  to  identify  individuals  in  need  of  further  evaluation  or 
treatment.  There  are  several  factors  that  can  influence  the  effectiveness  of  screening  for 
mental  health  problems  in  high  risk  populations,  such  as  the  attitudes  and  beliefs  of  the 
individuals  being  screened.  Some  of  these  factors  include  stigma,  attitudes  toward  self¬ 
disclosure,  help-seeking,  social  and  work  support  systems,  and  the  screening  environment. 

During  Vanderbilt's  previous  evaluation  of  the  PDHRA  process,  a  survey  was  administered  to 
examine  the  relationships  between  SM  attitudes  relevant  to  the  PDHRA  process  and  other  self- 
reported  SM  characteristics  The  SM  survey  as  originally  administered  is  attached  to  the  end  of 
this  appendix.  For  the  purposes  of  the  previous  evaluation,  the  items  in  the  survey  were 
categorized  into  nine  theory-based  scales.  The  nine  scales  showed  reasonable  internal 
reliability  within  each  scale  (see  Table  A.l).  The  scales  were  evaluated  using  the  Cronbach's 
coefficient  alpha  to  assess  internal  consistency  and  reliability.  Cronbach's  alpha  (Cohen,  1988, 
1992)  was  calculated  for  each  set  of  items,  interpreting  a  >0.80  as  satisfactory,  indicating  that  a 
scale  was  of  sufficient  length  and  that  the  items  appeared  to  be  measuring  similar  content 
(Nunnally  &  Bernstein,  1994) 

Table  A.l  Number  of  items  and  standardized  Cronbach's  alphas  for  SM  Survey  scales 


Number  of  Items 

Alpha  (Std.) 

Scale  1.  Post-deployment  support  and  help  seeking 

10 

.77 

Scale  2.  Unit  cohesion  for  personal  problems 

3 

.88 

Scale  3.  PDHRA  leadership  support 

5 

.77 

Scale  4.  PDHRA  self  disclosure 

3 

.90 

Scale  5.  Satisfaction  with  the  PDHRA  provider 

7 

.87 

Scale  6.  Awareness  of  others'  problems 

4 

.77 

Scale  7.  General  willingness  to  self-disclose 

4 

.75 

Scale  8.  Perceived  stigma  related  to  disclosure 

4 

.88 

Scale  9.  Barriers  to  accepting  mental  health  referral 

7 

.80 

To  further  explore  the  previously  developed  survey  for  the  purpose  of  the  current  project,  the 
factorial  validity  of  the  scales  were  evaluated  using  exploratory  factor  analysis  in  order  to 
further  validate  that  the  measurement  model  fits  the  theory  of  what  each  scale  purports  to 
measure.  In  exploratory  factor  analysis,  data  are  described  and  summarized  by  grouping 
variables  together  that  are  correlated  with  one  another.  These  groupings,  or  factors,  are  thus 
hypothesized  to  be  measuring  the  same  underlying  (latent)  processes  (Tabachnick  &  Fidell, 
2007).  The  results  of  this  analysis  are  presented  below.  Note  that  the  procedure  for  data 
collection  and  demographic  information  that  follows  has  already  been  presented  in  Vanderbilt's 
previous  evaluation  (Bickman,  et  al.,  2009),  and  is  repeated  here  for  descriptive  purposes. 

Methods 
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Data  Sources 

The  source  of  data  for  this  secondary  analysis  consists  of  an  attitude  and  support  assessment 
survey.  Surveys  were  collected  from  6,714  SMs  during  44  PDHRA  events  (see  Table  A. 2). 

Service  Member  Survey 

The  SM  survey  included  82  items,  53  of  which  were  measured  on  a  5  point  Likert  scale,  on 
factors  that  may  influence  the  PDHRA  process  (see  end  of  this  appendix  for  the  survey).  Items 
were  either  created  by  VU  or  selected  from  the  literature.  Items  pertaining  to  post-deployment 
social  support  were  taken  from  the  Deployment  Risk  and  Resilience  Inventory  (King,  King,  Vogt, 
Knight,  &  Samper,  2006)  and  the  Unit  Behavioral  Health  Needs  Assessment  Survey  (UBHNAS; 
(Department  of  Military  Psychiatry  Walter  Reed  Army  Institute  of  Research  [WRAIR],  2006). 
Items  from  the  UBHNAS  also  measured  barriers  to  care  and  stigma,  in  addition  to  items  created 
by  Hoge  and  colleagues  (Hoge,  et  al.,  2004).  Items  from  the  Self  Awareness  Assessment  were 
used  to  measure  self  awareness  (http://www.myskillsprofile.com/questionnaire.php).  Attitudes 
toward  help  seeking  were  measured  using  items  from  the  following  measures:  the  UBHNAS,  the 
Attitudes  Toward  Seeking  Professional  Psychological  Help  (Fischer  &  Turner,  1970)  and 
questions  developed  by  Vogel  and  colleagues  (Vogel,  Wade,  Wester,  Larson,  &  Hackler,  2007). 
Items  measuring  self  disclosure  were  obtained  from  the  Distress  Disclosure  Index  (Kahn  & 
Hessling,  2001). 

Questions  were  also  asked  about  the  PDHRA  process,  such  as  if  the  SM  knew  the  PDHRA 
provider  prior  to  the  interview.  Some  of  these  items  were  derived  from  a  satisfaction  survey 
created  by  a  contractor  to  the  Army,  Booz  Allen  Hamilton.  Other  individual  items  included 
questions  about  the  SMs  plans  for  retirement  or  seeking  promotion  and  whether  they  knew  the 
DoD  policy  on  health  disclosure.  Lastly,  the  SM  survey  included  questions  about  demographic 
background,  including,  age,  gender,  and  rank. 

Procedure 

SMs  from  each  of  the  four  military  branches  -  Army,  Navy,  Marines,  and  Air  Force  -  were 
eligible  to  participate  in  the  survey.  However,  survey  participants  were  recruited  in  different 
ways  and  at  different  times.  Service  members  from  the  Army,  Navy,  and  Marines  were 
recruited  during  Vanderbilt  University  (VU)  site  visits  to  ten  PDHRA  events  from  January  2009 
through  April  2009.  All  SMs  who  completed  the  PDHRA  during  the  site  visit  were  eligible  to 
participate  in  the  survey.  Participants  from  the  Air  Force  were  recruited  online  with  the 
assistance  of  the  Air  Force  Medical  Operations  Agency  within  four  weeks  of  completing  the 
PDHRA  process.  All  Air  Force  participants  completed  the  PDHRA  between  March  15,  2009  and 
May  15,  2009.  Other  SMs  completed  the  survey  in  conjunction  with  a  PDHRA  event  at  one  of  34 
traveling  team  events  held  by  a  contracted  agency  who  conducts  PDHRA  screenings  for  DoD; 
these  screenings  took  place  between  March  6,  2009  and  April  5,  2009.  For  a  complete  list  of 
participants  by  PDHRA  event  location,  see  Table  1. 

VU  researchers  visited  ten  locations  to  recruit  SMs  for  surveys  after  they  completed  the  PDHRA 
process.  In  order  to  link  the  survey  information  to  the  corresponding  PDHRA  forms,  SMs  were 
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asked  to  provide  their  birth  date,  initials,  branch  of  service,  and  pay  grade  on  blue  cards.  Each 
card  was  printed  with  a  unique  serial  number,  which  was  also  printed  on  the  survey.  The  cards 
were  separated  from  the  survey  and  sent  to  Force  Health  Protection  and  Readiness  (FHP&R) 
where  the  information  was  placed  in  a  spreadsheet  which  was  then  sent  to  an  epidemiologist 
at  the  Armed  Forces  Health  Surveillance  Center  (AFHSC)  who  had  access  to  PDHRA  files.  Each 
record  (i.e.,  SM)  in  the  data  set  was  assigned  a  unique  ID  number.  The  number  was  used  to  link 
the  SM  survey  data  and  the  PDHRA  data.  After  all  identifying  information  was  removed  from 
the  PDHRA  files,  they  were  sent  to  Vanderbilt  along  with  the  unique  ID  number.  Vanderbilt 
maintained  the  hard  copy  surveys  from  site  visits,  which  contained  no  identifying  information, 
when  the  visit  was  finished. 

The  procedure  described  above  relates  to  Army,  Navy,  and  Marine  site  visits.  The  linking 
procedure  for  Air  Force  participants  was  slightly  different  because  no  Air  Force  site  visits  were 
conducted;  data  were  obtained  with  cooperation  from  the  Air  Force  Medical  Operations 
Agency  (AFMOA).  For  the  Air  Force,  individuals  who  had  completed  the  PDHRA  recently  (March 
15  through  May  15,  2009)  were  recruited  to  complete  the  SM  survey  via  an  online  survey. 
AFMOA  provided  FHP&R  with  the  same  information  on  the  site  visit  cards  for  everyone 
recruited  to  participate,  along  with  a  unique  ID  number  assigned  to  each  SM.  FHP&R  and  AFHSC 
used  this  information  to  match  completed  surveys  to  the  corresponding  PDHRAs,  with  the 
resulting  de-identified  linking  file  sent  to  Vanderbilt. 

SMs  recruited  during  traveling  team  events  conducted  by  the  contracted  agency's  travelling 
team  did  not  use  the  above  linking  process  and  thus  were  not  able  to  be  linked  to  a  PDHRA.  The 
contracted  agency's  staff  distributed  and  collected  the  surveys,  and  sent  them  to  VU  at  the  end 
of  each  event. 

Note  that  198  SMs  who  had  been  deployed  to  Kosovo  and  were  surveyed  in  Johnston,  IA 
completed  the  SM  survey.  These  SMs  were  excluded  from  this  analysis  to  maintain  consistency 
in  deployment  locations  (i.e.,  all  other  participants  were  deployed  for  OIF  or  OEF).  Thus,  the 
final  dataset  included  a  total  of  44  settings  (i.e.,  locations)  based  on  the  location  of  the  PDHRA 
event  and  the  group  responsible  for  collecting  the  data:  34  traveling  team  events  (TT) 
conducted  by  the  contracted  agency,  9  VU  site  visits,  and  online  AF  participation.  Table  A.2 
shows  the  number  and  percentage  of  participants  associated  with  each  setting. 

Table  A.2  Total  participants  completing  SM  surveys  by  PDHRA  setting  (N=6,714) 


Setting 

N 

% 

Traveling  team  collection 

Ft.  Gordon,  GA 

8 

0.1 

Ft.  Belvoir,  VA 

16 

0.2 

Jacksonville,  FL 

21 

0.3 

Sacramento,  CA 

24 

0.4 

Cortland  Manor,  NY 

26 

0.4 

Camp  Pendleton,  CA1 

27 

0.4 

Schaumburg,  IL 

29 

0.4 
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Setting 

N 

% 

Little  Rock,  AR 

35 

0.5 

Lexington,  KY 

39 

0.6 

Alameda,  CA 

39 

0.6 

San  Diego,  CA 

40 

0.6 

Manistee,  Ml 

41 

0.6 

North  Little  Rock,  AR2 

43 

0.6 

Geneseo,  NY 

44 

0.7 

Wyoming,  Ml 

47 

0.7 

Cadillac,  Ml 

49 

0.7 

Bossier  City,  LA 

50 

0.7 

Lima,  OH 

53 

0.8 

Dowagiac,  Ml 

57 

0.8 

Camp  Pendleton,  CA2 

64 

1.0 

Annville,  PA 

66 

1.0 

Tucson,  AZ 

67 

1.0 

North  Little  Rock,  AR 

68 

1.0 

Sandusky,  OH 

78 

1.2 

Camp  Roberts,  CA 

87 

1.3 

Tiffin,  OH 

90 

1.3 

Murray,  KY 

91 

1.4 

North  Little  Rock,  AR 

93 

1.4 

Cleveland,  OH 

130 

1.9 

Barrigada,  GU 

150 

2.2 

Arkadelphia,  AR 

190 

2.8 

Walbridge,  OH 

204 

3.0 

Evansville,  IN 

529 

7.9 

Indianapolis,  IN 

1173 

17.5 

Vanderbilt  University  site  visit  collection 

Milwaukee,  Wl 

64 

1.0 

Quantico,  VA 

68 

1.0 

Camp  Pendleton,  CA 

102 

1.5 

Port  Hueneme,  CA 

136 

2.0 

Ft.  Drum,  NY 

140 

2.1 

San  Diego,  CA 

312 

4.6 

Ft.  Riley,  KS 

489 

7.3 

Ft.  Wayne,  IN 

501 

7.5 

Ft.  Campbell,  KY 

878 

13.1 

Online  collection 

Air  Force 

256 

3.8 

Total 

6714 

100.0 

Study  Population 

Basic  demographic  characteristics  of  the  SMs  are  presented  here  for  informational  purposes. 
Because  a  random  sampling  procedure  was  not  feasible  for  this  study,  these  data  are  not 
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representative  of  all  military  personnel,  or  military  personnel  in  a  particular  Service  Branch  or 
Component.  Population  characteristics  are  presented  for  informational  purposes  only  and  were 
not  used  to  analyze  group  differences. 

Branch  and  Component 

Table  A. 3  shows  that  survey  participants  served  with  the  four  main  military  branches  -  Army, 
Marines,  Air  Force,  and  Navy.  However,  the  percentage  associated  with  each  branch  is  highly 
variable.  The  vast  majority  of  SMs  (83%)  were  Army;  relatively  few  were  in  the  Navy  (2%)  or  Air 
Force  (4%).  The  sample  is  also  unevenly  distributed  by  component.  While  most  (59%) 
participants  were  in  the  National  Guard,  very  few  were  in  the  Reserve  (6%).  The  distribution  of 
participants  by  component  was  highly  associated  with  Service  Branch.  For  example,  100%  of 
participants  in  the  Air  Force  and  Navy  had  active  duty  status. 


Table  A. 3  Component  by  Branch 


Army 

N=5540 

Marines 

N=782 

Navy 

N=136 

Air  Force 

N=256 

Total 

N=6714 

Component 

National  Guard 

71.6% 

0.0% 

0.0% 

0.0% 

59.1% 

Active 

27.2% 

52.9% 

100.0% 

100.0% 

34.5% 

Reserve 

1.2% 

47.1% 

0.0% 

0.0% 

6.4% 

Total 

82.6% 

11.6% 

2.0% 

3.8% 

100% 

Rank/Grade 

As  Table  A.4  shows,  most  (51%)  participants  were  enlisted  with  a  pay  grade  of  E01  to  E04; 
about  a  third  were  E5-E6,  with  categories  by  rank/grade  decreasingly  represented  as 
rank/grade  increased.  While  this  pattern  was  found  across  all  branches  of  the  military, 
participants  in  the  Air  Force  tended  to  be  of  higher  rank  than  those  in  other  branches  of  service. 


Table  A.4  Rank/Grade  by  Branch 


Army 

N=5369 

Marines 

N=775 

Navy 

N=133 

Air  Force 

N=255 

Total 

N=6532 

Rank  or  Grade 

E1-E4 

50.2% 

66.1% 

60.9% 

23.1% 

51.3% 

E5-E6 

32.2% 

25.2% 

34.6% 

42.7% 

31.8% 

E7-E9 

7.6% 

4.1% 

2.3% 

16.5% 

7.4% 

01-03 

6.5% 

3.5% 

2.3% 

8.6% 

6.2% 

04-09 

1.7% 

1.2% 

.0% 

9.0% 

1.9% 

W01-W05 

1.7% 

.0% 

.0% 

.0% 

1.4% 

Table  A. 5  shows  that  participant's  rank  varied  by  Service  component.  For  example,  a  larger 
proportion  of  survey  participants  in  the  Reserves  had  lower  grade/rank  than  other  participants 
in  other  service  components. 
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Table  A.5  Rank  by  Service  Component 


Grade  or 

Rank 

Active 

N=2253 

National 

Guard 

N=3851 

Reserve 

N=428 

Total 

N=6352 

Rank  or  Grade 

E1-E4 

44.1% 

53.3% 

70.3% 

51.3% 

E5-E6 

34.1% 

31.6% 

22.2% 

31.8% 

E7-E9 

8.6% 

7.2% 

3.3% 

7.4% 

01-03 

8.2% 

5.4% 

2.3% 

6.2% 

04-09 

2.0% 

1.8% 

1.6% 

1.9% 

W01-W05 

3.0% 

0.6% 

0.2% 

1.4% 

Age 

While  most  (38%)  participants  were  less  than  25  years  old,  older  age  groups  were  well 
represented  (Table  A. 6).  However,  age  tended  to  vary  by  Service  component,  at  least  among 
survey  participants  (Table  A. 7).  Reservists  tended  to  be  younger  than  those  in  Active  Duty  or 
the  National  Guard  -  61%  were  between  18-24  years  old  compared  to  41%  and  34% 
respectively.  Participants  in  the  National  Guard  were  more  than  twice  as  likely  (15%)  as  others 
to  be  40  or  more  years  of  age. 


Table  A. 6  Age  by  Branch 


Army 

N=5519 

Marines 

N=781 

Navy 

N=136 

Air  Force 

N=254 

Total 

N=6690 

Age 

18-24 

35.3% 

62.2% 

61.0% 

19.3% 

38.3% 

25-29 

25.8% 

22.9% 

27.9% 

31.1% 

25.7% 

30-39 

25.7% 

12.7% 

10.3% 

32.3% 

24.1% 

40  or  over 

13.2% 

2.2% 

0.7% 

17.3% 

11.8% 

Table  A.7  Age  by  Component 


Active 

N=2303 

National 

Guard 

N=3954 

Reserve 

N=433 

Total 

N=6690 

Age 

18-24 

41.0% 

34.4% 

60.5% 

38.3% 

25-29 

28.4% 

24.6% 

22.4% 

25.7% 

30-39 

23.4% 

25.8% 

12.0% 

24.1% 

40  or  over 

7.2% 

15.2% 

5.1% 

11.8% 
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Gender 

While  most  participants  were  unsurprisingly  male  (92%)  in  all  branches  of  service,  females  were 
about  twice  as  likely  to  be  in  the  Air  Force  (20%)  or  Navy  (16%)  than  in  the  Army  (8%)  (Table 
A.8).  Gender  also  varied  by  Component.  While  nearly  all  (98%)  reservists  are  male,  a  larger 
proportion  of  females  are  active  duty  (11%)  or  in  the  National  Guard  (7%)  (Table  A. 9). 


Table  A.8  Gender  by  Branch 


Army 

N=5337 

Marines 

N=775 

Navy 

N=132 

Air  Force 

N=253 

Total 

N=6497 

Gender 

Male 

92.3% 

96.4% 

84.1% 

80.2% 

92.1% 

Female 

7.7% 

3.6% 

15.9% 

19.8% 

7.9% 

Table  A.9  Gender  by  Component 


Active 

N=2247 

National 

Guard 

N=3823 

Reserve 

N=427 

Total 

N=6497 

Gender 

Male 

89.2% 

93.3% 

97.7% 

92.1% 

Female 

10.8% 

6.7% 

2.3% 

7.9% 

Analyses 

SM  Survey  Scaling  Procedure 

The  total  sample  of  6,714  SMs  who  responded  to  the  survey  were  split  into  two  random  sub¬ 
samples  (Sample  One  n=3,316  and  Sample  Two  n=3,398)  for  replication  purposes.  Using  only 
the  data  from  sample  one,  all  of  the  5-scale  Likert  items  on  the  SM  survey  that  corresponded  to 
the  nine,  previously  defined,  theory-based  scales  were  entered  into  exploratory  factor  analysis 
(EFA)  with  oblique  rotation.  Inspection  of  Eigenvalues  allows  for  the  investigation  of  the 
number  of  factors  (or  dimensions)  the  individual  items  are  measuring.  Generally,  every 
eigenvalue  over  one  indicates  the  potential  of  a  unique  factor  being  measured.  Without 
restrictions,  the  EFA  yielded  10  factors  with  Eigen  values  greater  than  one.  Further  exploration 
of  these  factors  led  to  the  discovery  of  a  dublet  (a  factor  with  only  two  items  loading  on  to  it). 
The  same  EFA  procedure  was  repeated  with  the  additional  restriction  added  in  the  syntax  to 
allow  only  nine  factors.  This  process  was  repeated  until  dublets  no  longer  existed  and  cross 
loadings  (items  loading  on  multiple  factors)  were  minimized.  The  final  number  of  unique  factors 
observed  for  this  sample  was  eight. 

The  same  EFA  procedure  described  above  was  repeated  for  an  independent  sample,  sample 
two.  With  no  restrictions  placed  on  this  procedure,  the  second  sample  yielded  11  factors. 
However,  when  the  same  restrictions  and  iterations  of  analysis  were  applied  to  the  data,  the 
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final  results  also  observed  eight  unique  factors.  This  replication  in  an  independent  sample 
increases  our  confidence  that  the  SM  survey  is,  in  fact,  measuring  eight  unique  factors. 

Once  these  eight  factors  were  distinguished,  a  closer  look  revealed  that  one  factor  had  items 
with  conflicting  item  loadings,  indicating  a  factor  with  poor  quality  characteristics.  The  factor 
consisted  of  the  following  items: 

•  49.  (Referral  NOT  helpful  because...)  I  can  handle  problems  on  my  own  or  with  help 
from  family  or  friends 

•  24.  Emotional  problems  are  more  likely  to  be  solved  with  professional  help  than  by 
trying  to  solve  them  alone 

•  12.  I  admire  people  who  solve  their  own  problems  without  seeking  professional  help 
The  factor  loading  for  item  49  was  in  the  opposite  direction  of  the  other  two  items. 
Additionally,  the  factor  also  had  poor  reliability,  most  likely  due  to  the  reverse  recoding  of  item 
12.  For  these  reasons,  this  factor  was  eliminated  as  a  scale. 

Across  both  independent  samples,  factor  loading  results  were  extremely  similar;  in  fact,  49  of 
the  50  items  had  the  strongest  factor  loadings  on  the  same  corresponding  factors.  This 
increases  confidence  in  our  findings.  Items  with  dissimilar  item  loadings  or  that  display  poor 
qualities  were  identified  for  removal  from  scales.  Table  A.10  shows  the  items  that  were 
removed  from  the  scales  due  to  poor  quality  characteristics  such  as  (1)  factor  loadings  of  less 
than  0.3  across  both  samples;  (2)  Inconsistent  loadings  across  both  samples. 


Table  A.10  Items  removed  from  the  SM  Survey  scales  due  to  poor  characteristics 


Scale 

Description 

Items  Removed 

1 

Attitudes  about  self  disclosing  and  attitudes  after  self  disclosure 

"People  rarely  talk  to  me  about  their 
personal  problems" 

lor  5 

(1)  and  (5)  Awareness  of  other's  problems 

"1  am  aware  of  my  moods  and  feelings" 

6 

Barriers  to  accepting  mental  health 

"The  provider  seemed  out  of  touch  with 
what  it  was  like  to  be  deployed" 

The  final  factor  results  for  the  remaining  43  items  are  shown  in  Table  A. 11.  This  table  presents 
the  items  that  make  up  each  scale,  the  Cronbach's  internal  consistency  reliability  estimates, 
the  corresponding  factor  loadings  from  sample  one  and  two,  and  the  average  factor  loadings 
across  samples.  A  Cronbach's  alpha  greater  than  0.80  is  considered  satisfactory.  There  were  a 
total  of  seven  factors,  or  scales. 


Table  A.ll  EFA  factor  loadings  for  final  scales 


Factor  Loadings 

Sample  1 

Sample 

2 

Average 
Sample  1&2 

Scale  1:  Attitudes  about  self  disclosing  and  attitudes  after  self 
disclosure 

a =.844 

12  items 
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Factor  Loadings 

Sample  1 

Sample 

2 

Average 
Sample  1&2 

20.  1  prefer  not  to  talk  about  my  problems 

0.55 

0.68 

0.62 

13.  When  something  unpleasant  happens  to  me,  1  often  look 
for  someone  to  talk  to 

0.58 

0.65 

0.61 

15.  When  1  feel  depressed  or  sad,  1  tend  to  keep  those 
feelings  to  myself 

0.51 

0.64 

0.57 

23.  Among  my  friends  or  relatives,  there  is  someone  1  go  to 
when  1  need  good  advice 

0.53 

0.46 

0.49 

19.  1  am  carefully  listened  to  and  understood  by  family 
members  or  friends 

0.52 

0.43 

0.48 

8.  1  am  willing  to  tell  others  my  distressing  thoughts 

0.47 

0.45 

0.46 

14.  Among  my  friends  or  relatives,  there  is  someone  who 
makes  me  feel  better  when  1  am  feeling  down 

0.50 

0.39 

0.45 

9.  1  have  problems  that  1  can't  discuss  with  family  or  friends 

0.45 

0.44 

0.44 

16.  There  are  people  to  whom  1  can  talk  about  my 
deployment  experiences 

0.51 

0.36 

0.43 

17.  If  1  were  feeling  upset  or  down  for  a  long  time  1  would 
want  to  get  help 

0.43 

0.38 

0.40 

10.  If  1  thought  1  needed  it,  1  would  get  psychological 
counseling 

0.34 

0.30 

0.32 

11.  People  at  home  just  don't  understand  what  1  have  been 
through  while  in  the  Armed  Forces 

0.33 

0.30 

0.32 

Scale  2:  Unit  and  NCO  support 
a=.857 

8  items 

31.  (About  military  job)  If  1  had  an  emotional  or  family 
problem  someone  in  my  unit  would  figure  out  a  way  to  help 

me 

0.83 

0.81 

0.82 

30.  (About  military  job)  If  1  were  stressed  or  feeling  down 
someone  in  my  unit  would  be  supportive 

0.83 

0.80 

0.82 

33.  (About  unit  NCO)  Encourages  unit  members  to  be  open 
about  any  problems  they  might  be  experiencing  on  the  DD 
Form  2900 

0.72 

0.73 

0.72 

29.  (About  military  job)  The  members  of  my  unit  know  that 
they  can  depend  on  each  other 

0.70 

0.75 

0.72 

32.  (About  unit  NCO)  Makes  sure  that  there  is  time  to  attend 
appointments  for  physical,  mental,  or  dental  health 

0.66 

0.66 

0.66 

34.  (About  unit  NCO)  Strongly  supports  the  PDFIRA  process 

0.55 

0.58 

0.57 

36.  (About  unit  NCO)  Has  talked  about  his  or  her  own  service- 
related  mental  health  problems  or  treatment 

0.42 

0.33 

0.38 

35.  (About  unit  NCO)  Has  no  compassion  for  unit  members 
experiencing  emotional  or  family  problems 

0.34 

0.41 

0.38 

Scale  3:  PDHRA  self  disclosure  -  Honesty 
a =.905 

3  items 

57.  (1  fully  disclosed...)  Any  problems  or  concerns  about  my 
emotional  health 

0.94 

0.97 

0.96 

56.  (1  fully  disclosed...)  Any  problems  or  concerns  about  my 
physical  health 

0.84 

0.88 

0.86 
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Factor  Loadings 

Sample  1 

Sample 

2 

Average 
Sample  1&2 

58.  (1  fully  disclosed...)  Any  problems  or  concerns  about 
alcohol  use 

0.85 

0.86 

0.86 

Scale  4:  Satisfaction  with  PDHRA  provider 
a =.914 

6  items 

65.  1  felt  a  great  deal  of  trust  in  the  provider 

0.83 

0.89 

0.86 

66.  1  learned  a  lot  from  the  provider67.  Did  you  know  the 
provider  who  did  the  PDHRA  interview  before  this  contact? 

0.83 

0.85 

0.84 

64.  The  provider  helped  me  be  more  aware  of  my  problems 

0.84 

0.81 

0.82 

61.  The  provider  reviewed  my  health  in  adequate  detail 

0.78 

0.80 

0.79 

60.  The  provider  who  did  the  PDHRA  interview  showed 
interest  and  concern  for  my  well-being 

0.78 

0.79 

0.78 

63.  1  felt  the  provider  could  help  me  get  access  to  the  care  1 
need 

0.74 

0.71 

0.73 

Scale  5:  Awareness  of  others'  problems 
a =.774 

4  items 

26.  1  can  spot  the  signs  of  depression 

0.80 

0.75 

0.78 

25.  1  can  spot  the  signs  of  post-traumatic  stress  (PTSD) 

0.74 

0.72 

0.73 

27.  1  can  spot  the  signs  of  a  concussion 

0.65 

0.65 

0.65 

28.  1  know  what  to  look  for  to  determine  if  someone  is 
drinking  too  much  alcohol 

0.51 

0.57 

0.54 

Scale  6:  Barriers  to  accepting  mental  health  referral 
a =.803 

6  items 

44.  (Referral  NOT  helpful  because...)  It  would  cost  too  much 
money 

0.70 

0.74 

0.72 

43.  (Referral  NOT  helpful  because...)  It  would  be  too  hard  to 
get  time  off  work 

0.62 

0.64 

0.63 

46.  (Referral  NOT  helpful  because...)  The  services  provided  are 
not  effective 

0.59 

0.62 

0.61 

45.  (Referral  NOT  helpful  because...)  The  visit  would  not 
remain  confidential 

0.53 

0.54 

0.53 

47.  (Referral  NOT  helpful  because...)  The  medications  that  1 
might  be  given  have  too  many  bad  side  effects 

0.51 

0.43 

0.47 

48.  (Referral  NOT  helpful  because...)  Religious  counseling 
would  be  more  helpful  than  mental  health  treatment 

0.37 

0.28 

0.33 

Scale  7:  Perceived  consequences  of  self  disclosure 
a =.884 

4  items 

42.  (If  1  revealed  any  emotional  problems...)  My  unit 
leadership  would  have  doubts  about  my  dependability 

0.85 

0.79 

0.82 

41.  (If  1  revealed  any  emotional  problems...)  Members  of  my 
unit  would  have  less  confidence  in  me 

0.83 

0.78 

0.80 

40.  (If  1  revealed  any  emotional  problems...)  It  could  harm  my 

career 

0.79 

0.73 

0.76 

38.  (If  1  revealed  any  emotional  problems...)  1  could  be  denied 
a  security  clearance  in  the  future 

0.65 

0.61 

0.63 
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Table  A. 12  shows  the  correlations  among  the  seven  new  SM  survey  scales.  The  scales  were 
correlated  as  expected  and  were  similar  to  the  correlations  between  the  scales  developed  in 
the  previous  evaluation.  Note  that  all  correlations  were  statistically  significant;  more 
meaningful  however,  are  those  with  moderate  (0.30)  to  large  (0.50)  effect  sizes  (Cohen,  1988, 
1992).  These  are  bolded  in  the  table. 


Table  A.12  Correlations  among  newly  formed  SM  survey  scales 


1 

2  3 

4 

5 

6 

7 

Scale  1:  Attitudes  about  self  disclosing  and  attitudes  after 
self  disclosure 

- 

Scale  2:  Unit  and  NCO  support 

.47 

- 

Scale  3:  PDHRA  self  disclosure-  Honesty 

.26 

.23 

Scale  4:  Satisfaction  with  PDHRA  provider 

.37 

.39  T  .29 

Scale  5:  Awareness  of  others'  problems 

.30 

00 

T — 1 

o 

CO 

.19 

- 

Scale  6:  Barriers  to  accepting  mental  health  referral 

-.41 

1 

w 

(T> 

f— > 
00 

-.28 

-.11 

Scale  7:  Perceived  consequences  of  self  disclosure 

-.38 

-.33  -.14 

-.22 

-.09 

.54 

These  newly  formed  scales  changed  only  slightly  from  the  older,  theoretically-based  scales.  The 
new  analyses  provide  confidence  that  the  scales  are  now  more  robust.  For  the  purpose  of  this 
study,  the  new,  psychometrically-sound  scales  will  be  replace  the  use  of  individual  items  and 
previously  developed  scales. 
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DD-HA(OT)2345 

Today’s  Date: _ 

Location  Code: 


Instructions:  This  study  is  being  conducted  by  Vanderbilt  University,  which  has  been  contracted  by  the  DoD  to 
provide  an  evaluation  of  the  post-deployment  health  re-assessment  process  (PDHRA).  Tour  opinions  and  experience 
w  ill  help  the  military  improve  health  care  for  all  Service  Members.  This  questionnaire  Mill  take  about  15  minutes  or 
less  to  finish,  and  your  participation  is  voluntary.  You  can  skip  any  questions  or  refuse  to  answer  any  questions. 
Your  answers  will  remain  confidential  and  will  not  be  connected  to  who  you  are. 


The  PDHTL4  is  a  medical  screening  for  both  behavioral  health  and  physical  health  concerns.  It  is  to  help  you  access 
resources  for  any  concerns  you  may  ha\>e  about  your  health  after  returning  from  deployment.  The  PDHRA  consists  of 

a  self-report  questionnaire  called  the  DD  Form  2900  and  speaking  one-on-one  with  a  health  care  provider.  The 
PDHRA  typically  occurs  90  180  days  after  returning  from  deployment. 


18-24 
25-29 
30-39 
40  or  over 


Gender 
_ Male 

Female 


Grade  /  Rank 

_ E1-E4 

_ E5-E6 

F.7-F.9 

_  01-03 

_  04-09 

W01-W05 


Please  fill  in  the  blank  with  a  number  from  0  to  100°o. 

i.  What  percentage  of  the  Service  Members  returning  from  OIF/OEF  would  you  estimate  have  symptoms  of 
‘  Post-traumatic  Stress  Disorder  (P'l'SD)?  _ % 

ADSMO^S1,  ^at  Percentage  of  the  Service  Members  returning  from  OIF/OEF  would  you  estimate  received  a  deployment- 
related  concussion?  _ % 

For  the  remainder  of  the  questionnaire,  please  mark  your  answers  by  putting  an  X'  in  one  box  for  each 
numbered  item  If  you  would  like  to  change  an  answer  you  already  marked,  please  fill  in  the  entire  box  of  the 
incorrect  answer  and  mark  the  appropriate  answer  with  an  X’.  Only  answers  with  an  X’  will  he  recorded. 

Mark  ‘Yes '  or  “No  ’for  each  of  the  following  questions. 


ADSM03  3 

At  least  one  NCO  or  Officer  from  my  current  unit  was  in  theater  with  me  on  my  last 
deployment 

Yes  □ 

NoD 

ADSM04  4. 

At  least  one  unit  NCO  or  Officer  briefed  my  unit  on  the  PDHRA 

Yes  □ 

NoO 

ADSM05  5 

Are  you  planning  to  separate  from  the  military  in  the  next  6  months? 

Yes  1  1 

NoD 

ADSM06  6 

Are  you  seeking  promotion  within  the  military  in  the  next  6  months? 

Yes  □ 

No  □ 

The  following  questions  are  about  any  deployment  cycle  education  you  may  have  received  to  help  Service  Members 
reintegrate  post-deployment.  The  education  may  be  written  materials,  websites,  films,  or  videos  that  provide 
information  on  the  kinds  of  problems  that  Service  Members  might  face  post-deployment. 


For  question  7,  please  mark  ‘Yes’  or  ‘No’  in  the  1st  column,  and  if  YES, 
please  indicate  if  the  nmterials  were  “Helpful”  or  Not  Helpful”  in  the  2"* 
column. 

If  YF.S,  was  the  material 
helpful? 

?.  To  help  you  reintegrate  post-deployment,  did  you 

ADSM7A  a-  Read  any  written  materials? 

ADSM7B  b.  View  any  websites? 

ADSM7C  c.  See  a  film  or  video  not  on  the  Web? 

YesD  NoD 

Yes  □  NoD 
YesD  NoD 

ADSM7D 

Helpful  0  Not  Helpful  0 

ADSM2E 
Ilelplul  LI  Not  Helpful  1  1 

Helpful  0  Not  Helpful  1  1 

ADSN17F 
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ADSM08 

ADSM09 

ADSM10 

ADSM11 

ADSM12 

ADSM13 

ADSM14 

ADSM15 

ADSM16 

ADSM17 

ADSM18 

ADSM19 

ADSM20 

ADSM21 

ADSM22 

ADSM23 

ADSM24 

ADSM25 

ADSM26 

ADSM27 

ADSM28 

ADSM29 

ADSM30 

ADSM31 


How  much  do  you  DISAGREE  or  AGREE  with  the  statements  below... 

DD-I  IA(OT)2345 

A  either 

Agree 

$lrongl> 

nor 

Strongly 

Disagree  Dljagrce 

Disagree 

Agree  Agree 

g.  I  am  willing  to  tell  others  my  distressing  thoughts  Q  Q 

□ 

□ 

□ 

9.  I  have  problems  that  I  can't  diseuss  with  family  or  friends 

□ 

□ 

□ 

□ 

□ 

10.  If  I  thought  I  needed  il,  I  would  get  psychological  counseling 

□ 

□ 

□ 

□ 

□ 

n.  People  at  home  just  don't  understand  what  I  have  been 
through  while  in  the  Armed  Forces 

□ 

□ 

□ 

□ 

□ 

i2. 1  admire  people  who  solve  their  own  problems  without 
seeking  professional  help 

□ 

□ 

□ 

□ 

□ 

u.  If  something  unpleasant  happens  to  me,  I  often  look  for 
someone  to  talk  to 

□ 

□ 

□ 

□ 

□ 

i4.  Among  my  friends  or  relatives,  there  is  someone  who  makes 
me  feel  better  if  I  am  feeling  down 

□ 

□ 

□ 

□ 

□ 

is.  If  I  feel  depressed  or  sad.  I  tend  to  keep  those  feelings  to 
myself 

□ 

□ 

□ 

□ 

□ 

i6  There  are  people  to  whom  I  can  talk  about  my  deployment 
experiences 

□ 

□ 

□ 

□ 

□ 

i7.  If  I  were  feeling  upset  or  down  for  a  long  time  I  would  want 
to  get  help 

□ 

□ 

□ 

□ 

□ 

is  People  rarely  talk  to  me  about  their  personal  problems 

□ 

□ 

□ 

□ 

□ 

i9. 1  am  carefully  listened  to  and  understood  by  family  members 
or  friends 

□ 

□ 

□ 

□ 

□ 

2o  I  prefer  not  to  talk  about  my  problems 

□ 

□ 

□ 

□ 

□ 

2i.  I  am  aw  are  of  my  moods  and  feelings 

□ 

□ 

□ 

□ 

□ 

22. 1  don  t  allow  my  feelings  to  influence  my  decisions 

□ 

□ 

□ 

□ 

□ 

23.  Among  my  friends  or  relatives,  there  is  someone  I  go  to  if  I 
need  good  advice 

□ 

□ 

□ 

□ 

□ 

24.  Emotional  problems  are  more  likely  to  be  solved  with 
professional  help  than  bv  trying  to  solve  them  alone 

□ 

□ 

□ 

□ 

□ 

25. 1  can  spot  the  signs  of  post-traumatic  stress  (PTSD) 

□ 

□ 

□ 

□ 

□ 

26. 1  can  spot  the  signs  of  depression 

□ 

□ 

□ 

□ 

□ 

27. 1  can  spot  the  signs  of  a  concussion 

□ 

□ 

□ 

□ 

□ 

28  I  know  what  to  look  for  to  determine  if  someone  is 
drinking  too  much  alcohol 

□ 

□ 

□ 

□ 

□ 

How  much  do  you  DISAGREE  or  AGREE  with  the  statements  below,  about  your  military  job... 

29.  The  members  of  my  unit  know  that  they  can  depend  on  each 
other 

□ 

□ 

□ 

□ 

□ 

3o.  If  I  were  stressed  or  feeling  down  someone  in  my  unit  would 
be  supportive 

□ 

□ 

□ 

□ 

□ 

3i.  If  I  had  an  emotional  or  family  problem  someone  in  my  unit 
would  figure  out  a  way  to  help  me 

□ 

□ 

□ 

□ 

□ 
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How  much  do  you  DISAGREE  or  AGREE  with  the  statements  below... 

Neither 

Agree 


My  unit  NCO... 

Strongly 

Disagree 

Disagree 

nor 

Disagree 

Strongly 

Agree  Agree 

ADSVLT  32  Makes  sure  ^lal  •here  is  lime  to  attend  appointments  for 
physical,  mental,  or  dental  health 

□ 

□ 

□ 

□ 

□ 

AJDSVL33  33  Encourages  unit  members  to  be  open  about  any  problems 
they  might  be  experiencing  on  the  DD  Form  2900 

□ 

□ 

□ 

□ 

□ 

ADSM34  ^  strongiy  supports  the  PDHRA  process 

□ 

□ 

□ 

□ 

□ 

35.  Has  no  compassion  for  unit  members  experiencing  emotional 
A™5  or  family  problems 

□ 

□ 

□ 

□ 

□ 

jo  Has  talked  about  his  or  her  own  service-related  mental  health 

A 1  )S M36  problems  or  treatment 

□ 

□ 

□ 

□ 

□ 

37.  My  answers  to  the  above  questions  would  be  the  same  for 
ADSM37  my  unit  officer 

□ 

□ 

□ 

□ 

□ 

Ifl  were  to  reveal  any  emotional  or  mental  health  problems  on  a  PDHRA  it  is  likely  that... 

38. 1  could  be  denied  a  security  clearance  in  the  future 

□ 

□ 

□ 

□ 

□ 

ADSM39  35 '  ^  wou*d  assist  mc  *n  finding  the  help  I  need 

□ 

□ 

□ 

□ 

□ 

\nsM4i i 40,  ^ cou^  harm  my career 

□ 

□ 

□ 

□ 

□ 

ADSM41  "  Members  of  my  unit  would  have  less  confidence  in  me 

□ 

□ 

□ 

□ 

□ 

.  ...  My  unit  leadership  would  have  doubts  about  my 

ADSM42  dependability 

□ 

□ 

□ 

□ 

□ 

Being  referred  to  a  mental  health  provider  would  XOT he  helpful  because... 

ADSM43  «.  It  would  be  too  hard  to  gel  time  off  w  ork 

□ 

□ 

□ 

□ 

□ 

ADSY144  «•  It  would  cost  too  much  money 

□ 

□ 

□ 

□ 

□ 

ADSM45  «•  The  visit  would  not  remain  confidential 

□ 

□ 

□ 

□ 

□ 

ADSM46  ,|6.  The  services  provided  are  not  effective 

□ 

□ 

□ 

□ 

□ 

47.  The  medications  that  I  might  be  given  have  too  many  bad 
ADSM47  side  effects 

□ 

□ 

□ 

□ 

□ 

48.  Religious  counseling  would  be  more  helpful  than  mental 
ADSM48  health  treatment 

□ 

□ 

□ 

□ 

□ 

is.  I  can  handle  problems  on  mv  own  or  with  help  from  family 
ADSM49  ...  ,  1  •  1  - 

or  friends 

□ 

□ 

□ 

□ 

□ 

so.  Since  returning  from  your  last  deployment  have  vou  experienced  an  emotional,  alcohol,  stress, 
ADSM50  ,,  .  i — ii — i 

problem?  YcsQ  No[J 

or  family 

If  no.  skip  to  question  #51  If  yes.  have  you  talked  to  any  of  the  following  individuals  about  it? 

ADSM5<  t^oa. Medical  Professional  Yes  0  No  1  1 

At  )SMSOB-'ob. Mental  Health  Professional  Yes  d  No  1  1 

ADSM50Csoc.Religious  or  spiritual  leader  Yes  d  No  0 

AI)SM501)50d.Family  or  friend  Yes  d  No  1  1 

jS  |  si.  Have  any  friends  or  family  suggested  that  you  seek  help  from  a  professional  (such  as  a  counselor,  doctor, 
clergy,  etc.)  for  an  emotional,  alcohol,  stress,  or  family  problem?  Yes  d  No  1  1 
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The  following  questions  ask  about  your  experience  completing  the  PDHRA  process.  It  is  important  to  know  the 
degree  to  which  Service  Members  disclose  information  on  the  PDIIRA  so  that  we  can  understand  how  to  improve  the 
process.  Remember,  vour  answers  will  not  be  linked  to  vour  name  so  vour  responses  will  remain  confidential. 

ADSM52  32  Have  you  completed  the  PDHR  A  since  your  last  deployment?  ADSM52C  ADSM52D 


Yes,  on  a  computer  I  I 
ADSM52A 

If  no.  skip  to  question  #69 


Yes,  on  the  telephone  I  I 
ADSM52B 


Yes,  using  paper  and  pencil 


□ 


No  □ 


ADSM53  53  Where  were  you  when  you  completed  the  PDHRA  (DD  Form  2900)? 

ADSM53A  1 was  by  myself  Q  1  was  in  a  group  where  others  were  also  completing  the  form  Q  ADSN If  3 B 


ADSM54  5,1  When  I  completed  the  PDIIRA  (DD  Form  2900)  I  was. . .  On  duty  I  I 

ADSM54A 

How  much  do  you  DISAGREE  or  AGREE  with  the  statements  below... 


Off  duty  l~l 
ADSM54B 

A  either 


Agree 


Strongly 

Disagree 

Disagree 

nor 

Disagree 

Agree 

Strongly 

Agree 

ADSKFs  55  Completing  DD  Form  2900  helped  me  identify  my  concerns 

d 

d 

n 

d 

n 

On  the  self-report  questionnaire  (DD  Form  2900)  I fully  disclosed.. 
ADSMS6  5#-  Any  problems  or  concerns  about  my  physical  health  d 

d 

d 

d 

d 

ADSM57  57 ■  Any  problems  or  concerns  about  my  emotional  health 

d 

d 

d 

d 

d 

ADSM58  58.  Any  problems  or  concerns  about  alcohol  use 

d 

d 

d 

d 

d 

ADSM59  59,  After  completing  DD  Form  2900,  did  you  complete  a  one-on-one  interview  with  a  health  care  provider? 

Yes,  by  telephone  O  Yes,  in  person  d  Nod  ADSM59C 

ADSM59A  ADSM59B 

If  no,  skip  to  question  ft  69 


How  much  do  you  DISAGREE  or  AGREE  with  the  statements  below... 

Neither 

Agree 


Strongly 

Disagree 

Disagree 

nor 

Disagree 

Agree 

Strongly 

Agree 

ADSM60  60  provider  who  did  the  PDIIRA  interview  showed  interest 

and  concern  for  my  well-being 

□ 

□ 

□ 

□ 

□ 

ADSM61  “•  Ahe  provider  reviewed  my  health  in  adequate  detail 

□ 

□ 

□ 

□ 

□ 

at-non  ^  62  The  provi der  seemed  out  of  touch  with  what  it  is  like  to  he 

ADSM62  ,  , 

deployed 

□ 

□ 

□ 

□ 

□ 

ADSM63  43  ^  ^slt  the  provider  could  help  me  get  access  to  the  care  I  need 

□ 

□ 

□ 

□ 

□ 

ADSM64  41  Die  provider  helped  me  be  more  aware  of  my  problems 

□ 

□ 

□ 

□ 

□ 

ADSM65  65  I  felt  a  great  deal  of  trust  in  the  provider 

□ 

□ 

□ 

□ 

□ 

ADSM66  66  I  learned  a  lot  from  the  provider 

□ 

□ 

□ 

□ 

□ 

ADSM67  67'  Did  you  know  the  provider  who  did  the  PDHRA  interview  before  this  contact? 

Yes  □ 

NoD 

ADSM67A*7*  If  ves.  was  the  provider  associated  with  your  unit  when  you  were  deployed? 

Yes  □ 

NoD 

ADSM68  6s.  How  long  was  the  PDHRA  interview? 

Less  than  5  minutes  Q  3-10  minutes  Q  11-15  minutes 

16-25  minutes  Q 

26  minutes  or  more 

□ 

69  Did  you  know  that  current  DoD  policy  no  longer  requires  military  personnel  to  disclose 
service-related  mental  health  treatment  when  applying  for  security  clearance? 


ADSM69 
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Introduction 

This  review  of  the  literature  was  conducted  to  determine  the  structure,  content,  and 
effectiveness  of  several  strategies  used  to  improve  provider  communication  and  decision 
making.  This  includes  trainings  that  specifically  focus  on  communication  skills  and  interview 
style.  These  improved  communication  strategies  are  intended  to  create  better  relationships 
with  the  patients,  thereby  increasing  information  elicited  by  the  patient  and  improving 
compliance  with  medical  regimen.  However,  training  alone  does  not  produce  sustainable 
practice  behavior  changes.  The  effectiveness  of  training  can  be  maximized  with  the  addition  of 
ongoing  supports,  such  as  the  provision  of  decision  support  and  feedback  aids.  Studies  are 
summarized  that  describe  quality  indicators  of  feedback  and  decision  support  for  Post 
Deployment  Health  Reassessment  (PDHRA)  providers.  Focus  is  also  given  to  studies  that  have 
validated  or  utilized  the  behavioral  health  subscales  contained  in  the  PDHRA. 

Background 

Changing  provider  practice  behavior  is  not  an  easy  endeavor  (Bennett,  et  al.,  2000;  Davis,  1998; 
Davis,  et  al.,  1999).  Identifying  the  mechanisms  behind  change  is  complex,  as  multiple 
perspectives  are  often  needed  to  find  an  effective  common  ground  (Sexton  &  Kelley,  2010). 
Traditional  means  of  introducing  information  to  influence  behavior,  such  as  educational 
materials  and  workshops,  have  generally  not  resulted  in  long-term  provider  behavior  change  or 
patient  outcomes  improvement.  Dissemination  of  educational  guidelines  may  increase  provider 
access  to  information,  but  does  not  create  lasting  behavior  change  without  the  addition  of 
enabling  or  reinforcing  strategies,  such  as  feedback  (Veloski,  Boex,  Grasberger,  Evans,  & 
Wolfson,  2006).  To  effectively  learn  from  experience,  reliable  knowledge  about  the  success  or 
failure  of  performance  is  necessary.  Unfortunately,  the  environmental  cues  of  success  available 
to  providers  are  often  vague  and  unsystematic  (Bickman,  1999). 

In  Vanderbilt  University's  previous  evaluation  of  the  PDHRA  process,  it  was  concluded  that 
health  care  providers  who  conduct  PDHRAs  would  benefit  from  a  training  and  feedback 
intervention  (Bickman,  et  al.,  2009).  For  example,  the  relationship  between  multiple  PDHRAs 
completed  by  the  Service  members  (SMs)  for  the  same  deployment  was  explored.  Even  when 
the  process  was  completed  twice  within  one  week,  the  aggregated  correlations  within  the  first 
week  were  about  twice  as  high  for  the  SM  self-report  portion  (r=  0.88  for  PDHRA)  compared  to 
the  provider  portion  (r=0.46  for  PDHRA).  Thus,  when  SMs  are  presenting  similarly  on  different 
days  to  different  providers,  the  providers  are  documenting  different  risk  assessments,  major 
concerns,  and  referrals  (Bickman,  et  al.,  2009).  This  indicates  minimally  shared  approach  taken 
by  providers  and  indicates  a  need  to  increase  accuracy  and  reliability  of  the  clinical  interviews 
as  documented  on  the  PDHRA.  The  low  agreement  of  providers  is  not  likely  due  to  intervening 
health  care  or  changes  in  the  SMs'  health  status,  which  could  be  expected  to  be  somewhat 
stable  within  a  seven-day  period. 


49 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


Variability  can  be  introduced  to  a  clinical  interview  from  several  sources,  including  experience 
and  knowledge  of  the  health  care  provider,  interview  style  and  fact-finding  techniques,  context 
of  the  interview,  and  cognitive  biases,  errors  and  heuristics  in  clinical  decision  making  (van  Ryn, 
2002).  Substantial  variability  in  PDHRA  provider  interviews  was  discovered  during  the 
Vanderbilt  evaluation  (Bickman,  et  al.,  2009);  it  was  found  that  providers  treat  behavioral 
health  problems  differently  than  physical  health  problems.  For  example,  when  physical  health 
problems  were  endorsed  by  the  SM  on  the  self-report  providers  almost  always  asked  about 
previous  treatment  (87%  to  92%),  but  when  mental  health  symptoms  were  endorsed  providers 
were  much  less  likely  to  ask  about  previous  treatment  (50%  to  64%).  One  way  to  decrease  this 
variability  is  to  provide  decision  supports,  which  can  range  from  clinical  guidelines  (e.g.,  on 
utilization  of  self-report  items  and  provider  documentation)  to  interpretation  through  actuarial 
modeling  (e.g.,  Dawes,  Faust,  &  Meehl,  1989). 

Evidence  shows  the  clinical  interview  has  strengths  and  weaknesses  in  screening  and  is 
therefore  most  effective  when  combined  with  an  actuarial  process.  Research  over  the  last  70 
years  has  consistently  found  that  actuarial  decision  making  is  more  accurate  and  less  variable 
than  clinical  decision  making  in  most  cases.  In  a  review  of  617  comparisons  in  136  studies 
published  between  1920  and  1994,  Grove  et  al.  (2000)  found  only  eight  studies  in  which  clinical 
decision  making  surpassed  the  accuracy  of  actuarial  decision  making.  Several  factors  have  been 
cited  as  possible  explanations  for  the  superiority  of  actuarial  over  clinical  decision  making.  First 
and  foremost,  the  human  brain  is  not  efficient  at  noticing,  selecting,  categorizing,  retaining, 
retrieving,  manipulating,  and  appropriately  applying  information  for  the  purpose  of  making 
inferences  (Grove  &  Meehl,  1996).  As  a  result,  clinical  decision  making  is  prone  to  fluctuations 
in  judgment  due  to  the  influence  of  cognitive  errors,  application  of  heuristics,  and  biases 
(Dawes,  et  al.,  1989). 

Despite  the  weaknesses  of  clinical  interviews,  they  do  carry  inherent  strengths,  such  as  the 
ability  of  providers  to  probe  and  ask  follow-up  questions.  Clinical  decision  making  offers  a 
several  important  benefits  that  actuarial-only  decision  systems  simply  cannot  provide.  For 
example,  humans  can  notice  significant  exceptions  that  may  call  into  question  actuarial 
conclusions.  Moreover,  any  actuarial  formula,  like  the  clinical-decision  process,  will  lead  to  false 
positives  and  false  negatives.  Health  care  providers  are  in  a  unique  position  to  adjust  decision 
model  cutoffs  depending  on  reasoned  judgments  about  the  relative  consequences  of  false 
positives  and  false  negatives  for  a  given  individual.  In  addition,  health  care  providers  can 
collaborate  with  patients  to  make  treatment  decisions  that  take  into  account  individual  values 
and  preferences  (Frosch  &  Kaplan,  1999),  while  actuarial  systems  alone  cannot.  While  standard 
actuarial-based  assessments  can  improve  the  reliability  of  clinical  interviews,  the  overall  impact 
of  the  session  can  be  strengthened  based  on  the  correct  interpretation  and  use  of  the 
information  by  the  provider  (Flughes,  et  al.,  2000).  Health  care  providers  may  bring  to  bear 
advanced  knowledge  on  the  interpretation  of  the  "clinical  significance"  of  assessment  results  in 
light  of  contextual  variables. 
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Communication  Skills  Training 

The  self-reported  portion  of  the  PDHRA  provides  a  huge  amount  of  clinical  information,  but  the 
providers  who  interview  SMs  are  in  a  position  to  further  extract  information  from  the  SMs  that 
may  be  particularly  sensitive  and  thus  not  self-reported.  In  order  to  elicit  sensitive  information 
and  improve  the  encounter,  effective  provider  communication  skills  are  necessary.  Epstein  et 
al.  (1993)  defined  the  effectiveness  of  medical  encounters  broadly  in  terms  of  the  degree  to 
which  the  following  three  goals  are  achieved:  (a)  data  gathering,  (b)  patient  education,  and  (c) 
relationship  building.  A  fourth  goal  has  since  been  added  (Roter,  2002):  activating  and 
partnership  building.  Effective  communication  patterns  in  medical  encounters  can  facilitate 
each  of  these  goals  in  that  they  provide  a  balance  between  absolute  patient  autonomy  and 
what  may  be  seen  as  provider  paternalism.  Effective  communication  redistributes  interpersonal 
power  in  an  equitable  way,  serving  to  activate  partnership  building.  Effective  communication 
also  builds  rapport  and  increases  familiarity.  In  medical  encounters,  patients  and  physicians 
should  share  the  same  goals.  Interestingly  and  in  support  of  this,  primary  care  physicians  and 
their  patients  were  least  satisfied  with  "narrowly  biomedical"  encounters  compared  to  medical 
encounters  in  which  patient  values  were  respected  and  psychosocial  talk  was  more  prevalent 
(Roter,  et  al.,  1997). 

Research  has  consistently  found  that  effective  communication  (both  verbal  and  non-verbal)  is  a 
fundamental  requirement  for  the  provision  of  appropriate  health  services  (Collins,  et  al.,  2002). 
Interpersonal  interaction  and  communication  affects  patients'  attitudes,  responsiveness, 
adherence  to  health  care  provider  instructions,  and  subsequent  utilization  of  the  health  care 
system  (Collins,  et  al.,  2002;  van  Ryn,  2002).  For  this  reason,  training  in  communications  skills  is 
an  important  aspect  of  increasing  positive  patient  outcomes. 

Structure 

Intense  Trainings  Are  More  Effective  Than  Brief  Trainings 

A  positive  relationship  has  been  found  between  training  intensity  and  effectiveness  of  the 
training  (Rao,  Anderson,  Inui,  &  Frankel,  2007).  Many  effective  clinician  training  programs 
reviewed  were  moderate  to  high  intensity,  involving  at  least  one  day  of  initial  training 
(Fallowfield,  et  al.,  2003;  Levinson  &  Roter,  1993;  Rao,  et  al.,  2007).  In  fact,  some  studies  with 
extensive  trainings  reported  less  attrition  in  positive  behavior  and  an  increase  in  positive  effects 
(e.g.,  fewer  interruptions  and  more  summarizing  of  information)  more  than  a  year  after  training 
that  were  not  apparent  at  three  months  (Fallowfield,  Jenkins,  Farewell,  &  Solis-Trapala,  2003). 
Some  studies  have  shown  that  brief  trainings  can  be  effective  (Lewis,  Pantell,  &  Sharp,  1991), 
but  in  general  shorter  trainings  were  not  effective  or  less  effective  (Cheraghi-Sohi  &  Bower, 
2008;  Joos,  1996;  Levinson  &  Roter,  1993)  and  longer  trainings  were  usually  related  to  more 
positive  outcomes  (Ammentorp,  Sabroe,  Kofoed,  &  Mainz,  2007;  Fallowfield,  et  al.,  2003; 
Frostholm,  et  al.,  2005;  Lustig,  et  al.,  2001;  Roter,  et  al.,  1995).  In  addition  to  longer  training 
sessions,  consistent  follow-up  "boosters"  that  include  the  dissemination  of  data  on 
implementation  of  the  communication  skills  and  self-assessments  have  been  found  to  be 
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effective  (Lewis,  et  al.,  1991). 

Content 

Communication  Skills  Training  is  Related  to  Positive  Patient  Outcomes 

Effective  provider  training  programs  focus  on  a  few  core  components  such  as  content  training, 
communication  skills  training,  interview  style,  and  proper  documentation  techniques. 
Communication  skills  training  is  one  important  step  in  ensuring  that  health  care  providers 
become  better  at  performing  assessments.  While  experienced  providers  may  be  fluent  with 
content  (e.g.,  symptoms  related  to  behavioral  health  problems),  evidence  suggests  that 
interactions  can  be  enhanced  by  attention  to  interpersonal  communication  patterns  (e.g.  active 
listening).  Fortunately,  using  enhanced  communication  skills  doesn't  necessarily  increase  the 
length  of  the  visit  (Roter,  et  al.,  1997;  Roter,  et  al.,  1995)  unless  a  pre-visit  questionnaire  is 
administered  and  has  to  be  reviewed  by  the  provider  (Hornberger,  Thom,  &  MaCurdy,  1997). 

Due  to  a  lack  of  adequate  research  design,  outcomes  of  communication  skills  training,  including 
provider  behavior  change,  patient  satisfaction,  and  patient  outcomes,  tend  to  be  variable 
(Griffin,  et  al.,  2004;  Hulsman,  Ros,  Winnubst,  &  Bensing,  1999).  Despite  this  lack  of  consistency 
in  the  literature,  positive  results  have  been  discovered.  Providers  who  have  received  training  on 
interpersonal  communication  skills  provide  more  medical  counseling  (L.  D.  Brown,  et  al.,  2000), 
elicit  more  information  and  concerns  from  the  patient  (Joos,  1996;  Langewitz,  et  al.,  1998;  Rao, 
et  al.,  2007),  exhibit  greater  facilitative  communication  and  information  giving  (Kim,  et  al., 

2002;  Rao,  et  al.,  2007),  ask  more  open  ended  questions,  more  frequently  ask  opinions,  give 
more  biomedical  information,  have  less  negative  affect  (Levinson  &  Roter,  1993),  and  generally 
had  improved  communication  skills  (Fallowfield,  et  al.,  2003;  Rao,  et  al.,  2007).  They  were  also 
more  likely  to  receive  higher  patient  satisfaction  ratings  (Frostholm,  et  al.,  2005;  Rao,  et  al., 
2007)  than  providers  who  used  narrow  biomedical  communication  patterns  (Roter,  et  al., 

1997).  Trained  providers  were  also  more  likely  to  recognize  patients'  psychosocial  problems 
(Hulsman,  et  al.,  1999).  In  addition,  trained  providers  also  used  more  problem-defining  and 
emotion-handling  skills  and  elicited  more  psychosocial  problems  from  their  patients  (Roter,  et 
al.,  1995).  Finally,  trained  providers  screened  more  often  for  and  provided  more  information 
regarding  important  risk  factors  during  well  visits  (Lustig,  et  al.,  2001)  and  scored  higher  in 
clinical  proficiency  (Roter,  et  al.,  1995).  These  positive  effects  of  training  are  often  still  apparent 
after  a  year.  Even  after  twelve  months,  trained  providers  asked  fewer  leading  questions,  asked 
focused  and  open-ended  questions,  and  responded  well  to  patient  cues  (Fallowfield,  et  al., 
2003).  Providers'  own  self-confidence  improved  for  specific  communication  tasks  (Ammentorp, 
et  al.,  2007)  and  dealing  with  difficult  patients  (Stein,  Frankel,  &  Krupat,  2005). 

Patients  who  saw  providers  who  received  training  in  interpersonal  skills  communicated  more 
during  the  interaction  (Brown,  et  al.,  2000),  disclosed  more  medical  and  psychosocial 
information  (Brown,  et  al.,  2000;  Levinson  &  Roter,  1993),  and  perceived  receiving  more 
information  from  the  provider  (Joos,  1996;  Rao,  et  al.,  2007).  In  addition,  they  were  more 
satisfied  with  the  provider  (Brown,  et  al.,  2000)  and  were  less  emotionally  distressed  six  months 
later  (Roter,  et  al.,  1995).  While  there  is  not  sufficient  evidence  that  communication  skills 
training  is  related  to  a  significant  improvement  on  health  status  (Hulsman,  et  al.,  1999), 
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patients  of  trained  providers  had  less  negative  affect  and  showed  fewer  distress  signs  during 
the  visit  (Levinson  &  Roter,  1993). 

Interview  Style  Affects  Information  Elicited  by  Patient 

Context  describes  the  method  of  data  collection  (e.g.,  phone,  face-to-face)  and  interview  style, 
while  fact-finding  techniques  describe  the  general  verbal  and  non-verbal  strategies  used  by  the 
health  care  providers  to  elicit  and  clarify  information  during  the  interview  process.  Unless  the 
interview  style  and  fact  finding  procedures  are  standardized,  health  care  providers  may  vary 
widely  in  the  particular  strategies  they  adopt.  Studies  dealing  primarily  with  mental  health 
conducted  over  the  last  few  decades  have  found  that  clinical  interview  styles  can  have  a 
significant  impact  on  the  nature  of  information  elicited,  and  subsequent  diagnoses  rendered 
(Cox,  Holbrook,  &  Rutter,  1981;  Cox,  Rutter,  &  Holbrook,  1981;  Graham  &  Rutter,  1968; 
Hopkinson,  Cox,  &  Rutter,  1981;  Rutter  &  Cox,  1981;  Rutter,  Cox,  Egert,  Holbrook,  &  Everitt, 
1981;  Rutter  &  Graham,  1968).  Traditionally,  four  distinctive  styles  of  clinical  interview  have 
been  identified:  (1)  the  sounding  board  style  characterized  by  minimal  activity  on  the 
interviewer's  part;  (2)  the  active  psychotherapy  style  characterized  by  frequent  use  of 
techniques  to  elicit  feelings  from  the  interviewee;  (3)  the  structured  interview  style 
characterized  by  an  active  fact-oriented  technique  on  the  part  of  the  interviewer,  and  (4)  the 
systematic  exploratory  style  characterized  by  both  high  fact  and  high  feeling-oriented 
techniques.  Research  involving  the  direct  comparison  of  these  interview  styles  has  found  that 
health  care  providers  employing  the  two  fact-oriented  techniques  (structured  interview  and 
systematic  exploratory)  identified  more  symptoms  of  psychopathology  and  were  better  at 
identifying  negative  diagnoses  (Cox,  Rutter,  et  al.,  1981).  Other  studies  have  found  that  utilizing 
more  structured  interview  protocols,  in  general,  may  decrease  variability  in  psychiatric 
diagnosis  assigned  (Hughes,  et  al.,  2000;  Piacentini,  et  al.,  1993). 

Interactive  Trainings  Are  Effective 

A  recent  review  indicates  that  didactic  trainings  (e.g.,  typical  CME  workshops)  are  less  effective 
than  mixed  didactive  and  interactive  workshops  for  improving  health  care  provider  practice  and 
health  care  outcomes  (Forsetlund,  et  al.,  2009;  Roter,  et  al.,  1995).  The  use  of  role  play  and/or 
simulated  patients  (Lane  &  Rollnick,  2007),  audio  or  video  recordings  (Hulsman,  et  al.,  1999), 
and  provider  discussions  on  the  psychosocial  problems  encountered  in  practice  (Roter,  et  al., 
1995)  also  contributes  to  more  effective  training.  Effectiveness  of  training  can  also  be  enhanced 
by  accentuating  the  relationship  between  communication  skills  and  positive  patient  outcomes 
(Roter,  et  al.,  1995). 

Provider  Communication  May  Also  be  Targeted  Through  Feedback  and  Decision  Support 
Interventions 

Research  shows  that  trainings  specific  to  enhancing  communication  skills  are  of  use;  however,  it 
should  also  be  noted  that  interventions  to  address  provider  communication  patterns  may  also 
include  feedback  and  decision  support  components.  When  providers  engage  with  SMs  in  the 
interview,  they  are  making  decisions  about  what  to  say  and  how  to  say  it,  and  some  of  these 
communication  decision  points  may  be  amenable  to  improvement  by  the  provision  of 
information  or  reminders  during  the  interview.  While  a  clinical  reminder  to  "empathize"  is 
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unlikely  to  be  of  any  use,  Vanderbilt's  previous  study  identified  several  concrete 
communication  points  upon  which  a  decision  support  system  might  act.  First,  while  most 
providers  asked  SMs  a  general  question  about  their  physical  health  regardless  of  whether  any 
symptoms  were  listed  on  the  self  report  form,  far  fewer  asked  a  general  question  about  mental 
health  (Bickman,  et  al.,  2009).  Given  that  past  research  indicates  that  patients  may  find  such 
conditions  more  difficult  to  discuss  (Barney,  Griffiths,  Jorm,  &  Christensen,  2006),  it  is  all  the 
more  important  that  the  provider  give  the  SM  an  opening  to  discuss  these  symptoms,  even  if 
they  aren't  reported  on  the  self-report.  Vanderbilt's  previous  research  also  found  that  SMs 
were  more  likely  to  accept  referrals  if  the  provider  took  the  time  to  dispense  educational 
information  about  the  areas  of  concern  during  the  interview  (Bickman,  et  al.,  2009);  however, 
providers  only  offered  such  information  to  SMs  13.7%  of  the  time  for  mental  health  conditions 
and  8.9%  of  the  time  for  alcohol  concerns.  Even  when  the  SM  had  endorsed  a  problem  on  the 
self  report  form,  there  was  no  area  of  mental  or  physical  health  concern  for  which  providers 
offered  education  even  half  of  the  time  (Bickman,  et  al.,  2009).  These  findings  illuminate 
potential  communication  decision  points  that  are  ideal  targets  for  feedback  or  decision  support 
interventions  of  the  types  that  are  discussed  further  below. 

Feedback  and  Decision  Support 

Structure 

Feedback  is  Most  Effective  When  Provided  on  a  Recurring  Basis 

The  previous  section  of  the  literature  review  described  structural  and  content  elements  of 
effective  clinician  training  interventions,  while  noting  that  one-time  training  events  may  have 
limited  effects  on  clinician  behavior  in  the  long  term.  Lasting  provider  behavior  change  is  more 
likely  to  result  from  interventions  which  provide  actionable  information  to  providers  on  a 
regular  basis,  in  the  form  of  systematic  feedback.  Feedback  interventions  provide  systematic 
information  relevant  to  patient  outcomes,  a  potentially  powerful  tool  to  change  clinician 
behavior  that  has  been  successfully  applied  in  improving  outcomes  in  medicine  (e.g., (Davis,  et 
al.,  1999;  Duncan,  2000).  Such  feedback  may  take  a  variety  of  forms,  though  in  all  cases  it  will 
communicate  a  provider's  status  in  relation  to  a  particular  standard  of  care  and  will  serve  as  a 
means  for  the  recipients  of  feedback  to  identify  and  act  upon  areas  in  which  they  are  in  need  of 
improvement  (Veloski,  et  al.,  2006).  The  standard  that  the  feedback  is  designed  to  address  may 
be  derived  from  a  variety  of  sources.  In  many  cases,  it  may  be  a  professional  guideline,  but 
feedback  can  also  be  provided  on  the  basis  of  guidelines  or  statistical  norms  that  are  specifically 
relevant  to  the  patient  population  that  the  providers  routinely  see.  The  feedback  may  also  be 
delivered  in  a  variety  of  ways,  either  to  providers  individually,  or  to  groups  of  providers  as  units. 

Longer,  Professionally  Administered  Feedback  Interventions  Were  More  Effective 
Well-designed  feedback  interventions  are  generally  effective  in  improving  provider  behaviors. 

In  one  major  review  of  41  studies  (Veloski,  et  al.,  2006),  70%  of  feedback  interventions  were 
found  to  be  successful,  with  chances  of  further  improvement  when  certain  design  factors  were 
incorporated.  Providers  may  more  readily  accept  feedback  from  colleagues  or  individuals  who 
are  normally  part  of  their  professional  oversight  system.  Among  studies  in  which  this  occurred. 
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83%  were  successful,  as  opposed  to  50%  when  the  feedback  was  given  directly  to  providers 
from  the  research  teams.  This  point  informs  Vanderbilt's  decision  to  employ  a  "train  the 
trainer"  approach  regarding  the  provision  of  feedback  at  regular  meetings.  Studies  that  are  at 
least  a  year  long  also  tend  to  be  more  effective  than  those  of  shorter  duration  (Veloski,  et  al., 
2006).  While  the  timeline  for  the  proposed  pilot  is  of  short  duration,  positive  findings  for 
preliminary  outcomes  (e.g.,  feasibility,  acceptability  to  providers)  could  indicate  the  need  for  a 
follow-up  study  of  longer  duration. 

While  feedback  alone  can  serve  as  an  effective  means  of  improving  provider  behavior, 
additional  benefits  can  result  from  multi-faceted  interventions  such  as  the  one  intended  for 
Vanderbilt's  research.  Studies  have  been  conducted  in  which  feedback  had  been  combined  with 
some  other  intervention  intended  to  influence  behavior  (Veloski,  et  al.,  2006).  These  other 
interventions  were  divided  into  three  groups:  education  and  guideline  provision;  educational 
outreach  visits;  and  "other"  interventions.  For  studies  in  which  the  impact  of  education  and 
feedback  was  examined,  63%  reported  a  positive  impact— slightly  lower  than  the  70%  reported 
for  feedback-only  studies.  This  suggests  that  simple  training,  in  and  of  itself,  does  not 
necessarily  add  value  over  and  above  a  more  continuous  provision  of  feedback.  However,  some 
evidence  of  value  added  was  found  for  studies  that  examined  the  combination  of  feedback  with 
educational  outreach  visits;  in  this  subset,  75%  of  studies  reported  positive  results.  The  "other" 
category  included  a  wide  variety  of  interventions,  all  of  which  were  more  intensive  than  the 
provision  of  up-front  training  or  guidelines.  Examples  of  methodologies  used  in  this  category 
include  incorporation  of  algorithms,  reminders,  patient-mediated  interventions,  group 
consensus  processes,  and  multiple  other  strategies.  Among  this  admittedly  heterogeneous 
group,  81%  of  studies  reported  positive  effects  (Veloski,  et  al.,  2006).  This  is  the  highest  rate  of 
all  subsets  examined  and  provides  some  evidence  that  a  multi-faceted  approach  is  the  most 
likely  to  produce  positive  changes  in  provider  behavior. 

Decision  Support  Can  be  Used  to  Provide  Specific ,  Patient-tailored  Information  to  Provider 
The  feedback  described  so  far  has  been  of  a  type  that  generally  occurs  weekly  or  monthly, 
although  it  can  be  provided  according  to  any  recurring  cycle,  including  in  real  time.  In  a  decision 
support  system,  information  is  provided  to  the  clinician  in  real  time  (immediately  prior  to  or 
during  patient  encounters),  with  the  goal  of  improving  clinical  decision  making.  These  are  often 
computerized  systems  that  match  the  characteristics  of  individual  patients  to  a  computerized 
knowledge  base.  Algorithms  are  then  applied  to  generate  recommendations  that  are 
specifically  targeted  to  the  patient  (Garg,  et  al.,  2005).  Instead  of  spending  a  substantial  amount 
of  time  interpreting  multiple  fields  of  information,  the  provider  can  process  the  information  in  a 
short,  easy  to  read  format.  Patient  information  may  be  added  to  the  system  in  several  ways. 
Health  care  professionals  or  staff  may  enter  the  information  themselves,  or  patients  may  do 
so— for  example  by  filling  out  a  computerized  questionnaire  about  symptoms  they  are 
experiencing.  Alternatively  (or  additionally),  electronic  medical  records  can  be  linked  to  the 
decision  support  system  and  then  queried  to  retrieve  patient  characteristics  (Garg,  et  al.,  2005). 
The  information  from  the  computerized  knowledge  base  then  has  the  appropriate  algorithms 
applied  in  order  to  generate  recommendations  to  the  provider.  These  recommendations  may 
be  delivered  to  the  provider  through  the  patient's  electronic  medical  records,  by  pager,  or  via 
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printouts  that  can  be  placed  in  a  patient's  paper  chart  (Garg,  et  al.,  2005). 

Simple  Clinical  Reminders  Can  Improve  Provider  Behavior 

The  simplest  decision  support  tools  come  in  the  form  of  clinical  reminders,  which  are  usually 
computerized  popup  screens  that  appear  when  a  provider  accesses  medical  information  about 
a  patient.  Such  reminders  may  "pop  up"  only  after  the  system  accesses  information  in  the 
patient's  medical  records  and  screens  it  for  an  important  risk  factor,  or  they  may  appear  as 
general  reminders  of  recommended  practices.  The  VA  system  currently  uses  a  clinical  reminder 
system  as  part  of  its  protocol  for  identifying  patients  with  continuing  symptoms  of  traumatic 
brain  injury.  This  system  initially  pulls  very  basic  information  from  the  patient's  records;  if  the 
patient  presenting  had  a  military  separation  date  that  was  after  9-11-01,  the  TBI  Screening 
Clinical  Reminder  is  activated,  and  clinicians  are  directed  to  determine  whether  the  patient 
served  in  Operation  Enduring  Freedom  (OEF)  or  Operation  Iraqi  Freedom  (OIF).  If  not,  no  screen 
is  needed,  and  the  clinical  reminder  is  satisfied  for  that  patient  encounter.  If  the  patient  did 
serve  in  one  of  these  Operations,  however,  the  provider  administers  the  screening  tool  (Lew, 
2007). 

Although  simple,  this  type  of  support  at  the  time  the  provider  makes  a  decision  can  have 
immense  value  over  and  above  even  effective  weekly  or  monthly  feedback  provision.  Nair  et  al. 
(2010)  conducted  a  study  with  the  aim  of  identifying  the  type  of  intervention  that  would  result 
in  the  greatest  improvements  in  physician  behaviors  regarding  prophylactic  antibiotic 
administration  in  surgical  patients.  The  authors  compared  groups  of  physicians  who  were 
receiving  one  of  the  following  types  of  support:  standard  patient  anesthesia  records  that 
included  no  additional  information;  regular  email  feedback  that  informed  providers  of  instances 
in  which  they  had  failed  to  produce  appropriate  documentation;  monthly  summary  reports; 
and  real  time  electronic  alerts  in  the  context  of  a  computerized  decision  support  system. 
Physicians  receiving  email  alerts  exhibited  a  2.3%  positive  change  in  antibiotic  compliance. 
Those  who  received  summary  reports  improved  behavior  by  4.9%  and  those  who  received  the 
electronic  alerts  for  each  patient  improved  behavior  by  9.3%,  thus  achieved  greater  than  99% 
compliance  with  standard  guidelines  (Nair,  et  al.,  2010).  Thus,  while  regular  feedback  was 
beneficial,  support  at  the  time  of  decision  added  more  value.  Further,  while  this  was  a 
computerized  reminder  system,  it  is  possible  to  employ  clinical  reminders  in  a  paper-based 
fashion,  particularly  in  settings  like  those  typical  of  the  PDHRA  process.  Later  sections  of  this 
review  will  discuss  the  "decision  support"  clinical  reminders  that  are  already  available  to 
providers  on  the  PDHRA  form,  and  explore  possible  ways  of  creating  a  paper-based  system 
which  would  make  these  elements  more  comprehensive. 

More  Intensive  Decision  Support  Tools  Apply  More  Complex  Algorithms  to  Patient  Information 
More  complex  versions  of  decision  support  may  begin  with  clinical  reminders  but  then  go  on  to 
facilitate  provider  decision  making  in  more  complex  ways  by  feeding  additional  information 
from  patient  medical  records  into  the  decision  support  tool.  These  tools  may  apply  algorithms 
that  incorporate  information  from  several  fields  and  then  generate  a  recommendation  for  the 
provider.  For  example,  Kurian  et  al.  (2009)  reported  on  a  decision  support  system  that 
providers  used  to  help  guide  treatment  decisions  for  patients  with  depression.  In  this  system. 
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patient  symptoms,  medication  adherence  data,  and  experience  of  medication  side  effects  were 
components  of  an  algorithm  that  was  applied  to  generate  medication  recommendations  to 
providers.  Patients  whose  providers  used  the  system  reported  significantly  greater  symptom 
reduction  than  did  patients  whose  providers  employed  care  as  usual  (Kurian,  et  al.,  2009). 
Similar  systems  have  resulted  in  positive  provider  behavior  change  in  a  variety  of  other  areas, 
including  antibiotic  prescribing  (Nair,  et  al.,  2010),  cardiac  disease  management  (Santelices,  et 
al.,  2010;  Toth-Pal,  Wardh,  Strender,  &  Nilsson,  2008),  reduction  in  drug  dose  calculation  time 
(Balaguer,  Ballart,  &  Subias,  2001)  and  increasing  rates  of  osteoporosis  screening  (Dejesus,  et 
al.,  2010). 

While  a  computerized  decision  support  system  for  the  PDHRA  process  may  not  be  feasible  at 
this  time,  the  PDHRA  is  a  perfect  example  of  a  health  care  encounter  for  which  this  type  of  tool 
could  be  valuable.  Providers  conducting  PDHRA  interviews  have  a  very  limited  amount  of  time 
in  which  to  screen  for  a  variety  of  physical  and  mental  health  encounters.  In  Vanderbilt's 
previous  research,  some  providers  noted  that  interviewers  do  not  have  access  to  valuable 
information  on  the  Post-Deployment  Health  Assessment  (PDHA)  which  could  be  useful  in 
clarifying  risk  for  certain  conditions  (e.g.,  combat  exposure,  previous  self-reported  problems) 
(Bickman,  et  al.,  2009).  Further,  multiple  scales  are  employed  on  the  PDHRA,  with  no  current 
algorithms  in  place  to  help  providers  assess  risk  in  complex  situations  such  as  those  in  which  a 
SM  has  no  positive  screens  but  scores  just  below  the  threshold  for  multiple  conditions.  A 
computerized  decision  support  system  might  pull  in  relevant  PDHA  data,  calculate  individual 
subscale  results  automatically,  and  then  report  this  information  in  a  format  that  allowed 
providers  to  see  all  of  it  at  once.  Further  support  could  be  provided  by  the  inclusion  of 
additional  evidence-based  algorithms  that  providers  could  use  to  help  decide  whether  a  referral 
was  appropriate. 

Decision  Support  is  Effective  in  Producing  Provider  Change 

In  recent  years,  several  groups  of  researchers  have  conducted  comprehensive  reviews  of 
studies  that  examined  the  effectiveness  of  clinical  decision  support  systems.  The  largest  review 
to  date  was  conducted  by  Garg  et  al.  (2005),  who  reviewed  97  studies  that  compared  provider 
behavior  when  a  decision  support  system  was  used  to  behavior  of  providers  acting  without 
these  systems  in  place.  Positive  outcomes  were  reported  in  64%  of  these  studies,  and  these 
successful  studies  tended  to  share  important  features.  Most  importantly,  improvements  in 
behavior  were  more  likely  to  occur  when  providers  were  automatically  prompted  to  engage 
with  the  system,  as  opposed  to  situations  in  which  users  had  to  access  the  system  themselves. 
Improvements  occurred  in  73%  of  the  studies  that  included  automatically  activated  systems 
compared  to  47%  of  the  studies  in  which  providers  had  to  engage  the  systems  manually  (Garg, 
et  al.,  2005). 

There  are  at  least  three  additional  variables  that  are  associated  with  decision  support 
interventions  that  succeed  in  improving  provider  behaviors.  These  include  the  provision  of 
recommendations  rather  than  simple  assessments;  the  provision  of  decision  support  at  the 
actual  time  and  location  of  decision  making;  and  the  delivery  of  the  decision  support  through 
the  use  of  a  computerized  system.  One  review  found  that  94%  of  studies  that  combined  these 
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three  factors  with  an  automatically  activated  system  reported  positive  results  (Kawamoto, 
Houlihan,  Balas,  &  Lobach,  2005). 

Several  other  factors  influence  the  chances  that  a  decision  support  system  will  be  successful  by 
improving  the  likelihood  that  providers  will  willingly  accept  the  modifications  to  their  normal 
routines.  Representatives  from  military  installations  have  told  Vanderbilt  researchers  that  they 
don't  want  to  be  treated  like  robots  (Bickman,  et  al.,  2009),  and  indeed,  measures  to  reassure 
providers  that  their  professional  autonomy  is  not  threatened  are  essential  to  the  development 
of  a  system  that  will  not  be  resented  (Moxey,  et  al.,  2010).  It  can  be  helpful  to  have  new 
systems  endorsed  by  colleagues  prior  to  implementation,  and  careful  attention  to  interface 
design  can  ensure  that  the  provider's  ability  to  communicate  with  patients  is  not  compromised 
(Moxey,  et  al.,  2010).  In  fact,  a  system  that  efficiently  organizes  information  that  providers  view 
as  important  may  actually  make  it  possible  for  providers  to  spend  less  time  shuffling  through 
papers  looking  for  information  and  more  time  engaging  with  the  patient. 

Feedback  and  Decision  Support  Systems  are  Relevant  to  Military  and  VA  Clinical  Processes 
A  search  of  the  literature  did  not  reveal  any  intervention  that  examined  the  effectiveness  of 
training,  feedback,  or  decision  support  on  improving  provider  behaviors  during  the  PDHA  or 
PDHRA.  However,  relevant  studies  have  been  conducted  in  the  Veterans  Affairs  (VA)  system 
and  at  military  bases.  It  will  be  useful  to  describe  these  briefly  for  the  purposes  of  showing  that 
decision  support  systems  can  be  implemented  in  these  settings  with  positive  results. 

In  the  VA,  the  utility  of  a  decision  support  system  that  encouraged  providers  to  conduct  brief 
interventions  (Bis)  with  veterans  who  had  screened  positive  on  the  AUDIT-C  was  assessed 
(Lapham,  et  al.,  2010).  Providers  improved  their  behavior  from  baseline  after  they  received 
basic  education  on  this  topic,  but  far  greater  improvements  were  achieved  after  the 
implementation  of  an  electronic  clinical  reminder  (Lapham,  et  al.,  2010)  The  findings  of  this 
study  are  particularly  applicable  to  the  PDHRA  process,  as  the  reminder  issued  to  providers  is 
not  the  typical  directive  to  conduct  a  screening,  but  a  reminder  to  the  provider  about  what 
should  be  done  following  the  positive  screen. 

A  more  extensive  computerized  decision  support  system  has  been  tested  at  military  base 
practices,  and  has  resulted  in  modest  improvements  in  the  administration  of  screening  and 
preventive  measures,  though  not  in  improvements  in  acute  care  processes  (Apkon,  et  al., 

2005).  Further,  while  no  research  has  been  published  on  the  incorporation  of  decision  support 
into  the  PDHRA,  one  study  has  addressed  relevant  concerns  in  the  context  of  the  Pre¬ 
deployment  Health  Assessment  (PreDHA)  screening  questionnaire.  This  study  found  that  the 
self  report  form  had  low  validity  for  certain  items— in  this  case  self-identification  of  SMs  with 
previously  diagnosed  mental  health  disorders.  The  authors  of  the  study  conclude  that  an 
electronic  decision  support  system  that  incorporated  relevant  information  from  electronic 
health  records  would  facilitate  the  identification  of  SMs  at  high  risk  for  mental  health  disorders 
in  the  pre-deployment  period  (Nevin,  2009).  While  the  literature  on  decision  support  in  military 
settings  is  limited,  the  information  available  suggests  that  such  a  system  is  feasible  to 
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implement  and  likely  to  be  effective,  particularly  if  it  targets  provider  behavior  during  screening 
events  such  as  the  PDHRA. 

Content:  Actionable  Information  to  Inform  Feedback  Development  for  PDHRA  Providers 

Introduction 

While  development  of  appropriate  structure  and  delivery  is  essential  to  the  success  of  any 
feedback  intervention,  it  is  not  useful  in  the  absence  of  strong,  actionable  content.  Relevant 
content  could  be  derived  from  several  areas,  including  published  guidelines,  Vanderbilt's  own 
analysis  of  previous  PDHRA  data,  and  input  from  the  project's  Expert  Panel.  As  noted  in  the 
body  of  the  report,  Vanderbilt  has  not  yet  received  data  for  secondary  analysis.  However,  the 
research  team  has  reviewed  preliminary  findings  from  the  previous  research  contract  and  also 
explored  the  literature  relevant  to  five  behavioral  health  symptom  subscales  of  the  PDHRA 
process:  Posttraumatic  stress  disorder  (PTSD),  depression,  relationship  problems.  Traumatic 
Brain  Injury  (TBI),  and  alcohol  use. 

For  PTSD  and  depression,  previous  studies  have  validated  the  scales  in  forms  identical  or  very 
similar  to  those  that  are  used  on  the  PDHRA.  Consequently,  the  literature  review  for  these  two 
sections  summarizes  these  studies,  and  comments  upon  important  points  relating  to  the 
sensitivity  and  specificity  of  the  scales.  Findings  focus  on  sensitivity  and  specificity.  The 
sensitivity  of  a  model  is  its  ability  to  identify  those  at  risk,  while  specificity  refers  to  a  model's 
ability  to  identify  those  not  at  risk  of  the  outcome.  In  general,  sensitivity  and  specificity  are 
measures  that  assess  the  validity  of  diagnostic  and  screening  tests.  Practically  speaking,  a  highly 
sensitive  assessment  means  one  in  which  a  large  percentage  of  the  population  is  classified 
correctly  as  having  the  disorder;  a  highly  specific  assessment  is  one  in  which  individuals  without 
the  disorder  in  question  are  not  incorrectly  identified  as  having  the  disorder.  An  ideal  screening 
assessment  would  be  maximally  sensitive  and  specific,  with  100%  of  individuals  at-risk  being 
detected  and  100%  of  those  not  truly  at  risk  being  ruled  out.  This  framework  is  useful  for 
evaluating  decision  rules  or  cut  points  for  a  measure  because  it  accurately  reflects  how  an 
increase  on  any  one  of  these  indices  tends  to  co-occur  with  a  decrease  on  another. 

It  was  possible  to  report  fairly  extensively  on  sensitivity  and  specificity  values  for  the  PTSD  and 
depression  subscales.  For  the  relationship,  TBI,  and  alcohol  subscales,  however,  peer-reviewed 
validation  studies  of  the  items  as  they  appear  on  the  PDHRA  have  not  been  published. 
Consequently,  these  sections  of  the  literature  review  examine  those  scales  more  qualitatively. 

In  all  cases,  the  derivation  of  the  scales,  the  recommended  scoring  procedures,  and  the 
expectations  for  documenting  results  on  the  provider  portion  of  the  PDHRA  form  are  discussed. 
Finally,  after  the  separate  discussions  of  the  five  subscales,  this  review  concludes  with  a  more 
targeted  discussion  of  ways  in  which  a  feedback  intervention  might  help  improve  provider 
decision  making  in  instances  where  SMs  exhibit  symptoms  across  multiple  subscales. 

PDHRA  Subscales 

The  Post  Traumatic  Stress  Disorder  (PTSD)  Subscale 

The  PTSD  subscale  as  written  in  DD  Form  2900  is  taken  verbatim  from  the  well-validated 
Primary  Care  PTSD  Screen  (PC-PTSD)  (see  Prins  &  Oimette,  2004).  Any  study  that  conducted  a 
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validation  of  the  PC-PTSD  subscale  against  a  gold  standard  was  included  in  this  review.  Nine 
such  studies  were  found,  though  they  were  variable  in  regards  to  the  quality  of  the  gold 
standards  used  and  the  similarity  of  the  study  populations  to  the  PDHRA  population.  This 
variability  in  study  populations  is  smaller  than  it  might  be,  however,  because  the  PC-PTSD  was 
developed  in  the  VA  primary  care  setting,  and  consequently  almost  all  validation  studies 
conducted  on  the  scale  have  involved  veteran  or  military  populations.  Table  B.l  summarizes  the 
methodologies  used  in  each  study  along  with  basic  findings,  which  are  also  discussed  more 
generally  in  the  paragraphs  to  follow. 

All  of  the  studies  reviewed  established  that  endorsement  of  either  2  out  of  4  or  3  out  of  4 
symptoms  should  be  considered  a  positive  screen  for  PTSD  if  the  goal  is  to  achieve  optimal 
balance  of  sensitivity  and  specificity.  Most  authors  also  acknowledged  that  either  of  these 
cutoffs  could  be  appropriate  depending  on  whether  circumstances  favored  a  need  for  greater 
sensitivity  or  specificity  (Bliese,  et  al.,  2005;  Bliese,  et  al.,  2008;  Bliese,  Wright,  Adler,  &  Thomas, 
2004;  Calhoun,  et  al.,  2010;  Gore,  Engel,  Freed,  Liu,  &  Armstrong,  2008;  Prins  &  Ouimette, 

2004).  These  scoring  recommendations  were  based  on  assessments  of  the  sensitivity  and 
specificity  values  associated  with  the  various  possible  scoring  algorithms,  the  values  of  which 
varied  substantially  across  studies.  Reported  sensitivities  ranged  from  .45-. 97  when  2/4  items 
were  endorsed  and  .46-  .91  when  3/4  were  endorsed,  while  specificity  ranged  from  .57-. 96 
when  2/4  items  were  endorsed  and  .72-. 97  when  3/4  items  were  endorsed  (See  Table  B.l  for 
values  identified  in  individual  studies)  (Bliese,  et  al.,  2005;  Bliese,  et  al.,  2008;  Bliese,  et  al., 

2004;  Calhoun,  et  al.,  2010;  Gore,  et  al.,  2008;  Kimerling,  Trafton,  &  Nguyen,  2006;  Ouimette, 
Wade,  Prins,  &  Schohn,  2008;  Prins  &  Ouimette,  2004).  For  scoring  of  the  PDHRA, 
recommendations  released  by  USAMRU  researchers  suggest  scoring  a  2  as  positive  (Bliese,  et 
al.,  2008;  Wright,  Adler,  Bliese,  &  Eckford,  2008),  thus  choosing  to  err  on  the  side  of  sensitivity. 
This  represents  a  change  from  earlier  reported  recommendations  for  identifying  the  cutoff 
point  as  3  (Bliese,  et  al.,  2004).  In  spite  of  the  fact  that  these  guidelines  have  been  published,  no 
scoring  instructions  are  included  on  the  actual  DD  2900  form,  and  there  is  no  place  in  the 
provider  reporting  area  to  specifically  record  the  score  of  this  screen  as  providers  are  expected 
to  do  for  the  alcohol  and  TBI  subscales  (DD  2900  (PDHRA)  -  January  2008  Version,  2008).  It  is 
unknown  the  extent  to  which  providers  are  consistent  in  evaluating  the  four  questions 
contained  in  this  screening  subscale. 

Since  the  three  studies  by  Bliese  and  colleagues  are  the  only  ones  that  were  conducted  among 
active  SMs  in  a  post-deployment  context,  some  attention  to  these  findings  in  relation  to  those 
of  other  authors  may  be  useful  here.  These  studies  report  specificity  findings  that  are 
consistently  high,  especially  when  a  positive  score  is  defined  as  endorsement  of  3/4  items 
(Bliese,  et  al.,  2005;  Bliese,  et  al.,  2004;  Bliese,  et  al.,  2008).  One  possible  explanation  for  this  is 
the  fact  that  the  army  study  populations  were  made  up  of  individuals  taking  a  required,  non- 
anonymous  screening,  the  results  of  which  could  be  reported  to  commanding  officers.  This 
context  might  result  in  a  higher  rate  of  participants  underreporting  mental  health  symptoms 
during  the  screenings— a  circumstance  which  would  increase  specificity  (Calhoun,  et  al.,  2010). 
While  the  PDHRA  cannot  be  made  anonymous,  improvements  in  provider  communication 
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might  help  mitigate  some  of  the  concerns  SMs  have  about  reporting  symptoms. 


Table  B.l  Summary  of  Sensitivity  and  Specificity  Values  for  the  PC-PTSD 


Article 

N 

Study  Population 

Gold  Standard 

Sensitivity/ 

Specificity 

2/4  Endorsed 

Sensitivity/ 

Specificity 

3/4  Endorsed 

Bliese  et  al., 
2004 

592 

Active  Service  Post¬ 
deployment,  Army 

Adapted  MINI 

0.73 

0.88 

0.46 

0.97 

Bliese  et  al., 
2005 

367 

Active  Service  Post¬ 
deployment,  Army 

Adapted  MINI 

0.79 

0.78 

0.55 

0.92 

Bliese  et  al., 
2008 

352 

Active  Service  Post¬ 
deployment,  Army 

Adapted  MINI 

0.85 

0.76 

0.76 

0.92 

Calhoun  et 
al.,  2010 

220 

Recent  Veterans  who 
served  since  9/11/2001 

Structured 

Clinical  Interview 

for  DSM-IV  Axis 

1  disorders 
(SCID-1) 

0.89 

0.75 

0.83 

0.85 

Gore  et  al., 
2008 

213 

Patients  at  military 
primary  care  clinics  in 
Washington  DC 
Active  Service  (62%), 
Retired  Military  (18%), 
Family  of  Military(19%) 

Structured 

clinical  interview 
(unnamed) 

0.91 

0.71 

0.79 

0.85 

Kimerling  et 
al.,  2006 

97 

Veterans  being  treated 
for  substance  abuse  at 

VA  center 

Clinician 

Administered 

PTSD  Scale 
(CAPS) 

0.97 

0.57 

0.91 

0.80 

Ouimette  et 
al.,  2008 

11,230 

Veterans  seeking 
primary  care  identified 
through  records 

ICD-9  (no 
interview) 

0.45 

0.96 

No 

data 

No 

data 

Prins  et  al., 
2004 

188 

VA  General  Med/VA 
women's  health  clinics 

CAPS 

0.91 

0.72 

0.78 

0.87 

Van  Dam  et 
al.,  2010 

142 

Non-military/VA 
patients  already  in 
treatment  for 

Substance  Abuse 
Disorders  (Netherlands) 

SCID-I 

0.86 

0.57 

0.67 

0.72 

Note:  Bold  font  in  sensitivity  and  specificity  columns  denotes  the  algorithm  recommended  by  the  study's  authors. 

The  Depression  Subscale 

The  depression  screen  used  on  the  PDHRA  comes  from  the  Patient  Health  Questionnaire-2 
(PHQ-2),  which  consists  of  the  first  two  items  of  the  PHQ-9  (Spitzer,  Kroenke,  &  Williams,  1999). 
Responses  are  selected  from  a  4-item  Likert  Scale,  so  that  each  question  may  receive  a  score  of 
0  to  3  (Spitzer,  et  al.,  1999). 

The  version  of  the  PHQ-2  used  on  the  PDHRA  is  very  similar  to  the  original  instrument; 
however,  unlike  the  PTSD  screen,  some  differences  from  the  original  do  exist.  The  original 
questions  ask  respondents  about  symptoms  that  have  occurred  in  the  past  two  weeks,  while 
the  PDHRA  version  asks  about  the  past  month.  Otherwise,  wording  is  verbatim,  although  the 
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PDHRA  employs  a  unique  scoring  algorithm.  Outside  of  PDHRA  usage,  it  is  usually 
recommended  to  score  the  two  PHQ-2  questions  additively,  so  that  a  total  score  of  3  (out  of  a 
possible  6)  on  the  2  questions  is  classified  as  positive  (though  some  recommend  a  cutoff  of  2) 
(Arroll,  et  al.,  2010;  Bliese,  et  al.,  2005;  Chen,  et  al.,  2009;  Corson,  Gerrity,  &  Dobscha,  2004; 
Cutler,  et  al.,  2007;  Kroenke,  Spitzer,  &  Williams,  2003;  Lowe,  Kroenke,  &  Grafe,  2005; 
Richardson,  et  al.,  2010;  Thombs,  Ziegelstein,  &  Whooley,  2008)(See  Table  B.2  for  the  scoring 
rules  favored  in  each  study).  The  PDHRA  algorithm  is  slightly  different;  the  4-point  Likert  scale  is 
retained,  but  the  scores  from  the  two  questions  are  not  added.  A  positive  screen  is  an 
endorsement  of  either  (not  both)  of  the  two  questions,  and  it  is  usually  recommended  to  count 
a  score  of  2  or  above  as  positive  for  each  question.  Consequently,  any  SM  who  scores  at  least  a 
two  on  either  item  will  screen  positive  (Wright,  et  al.,  2008).  In  most  cases,  the  two  scoring 
algorithms  should  identify  cases  consistently.  However,  exceptions  to  this  exist,  as  in  the  two 
examples  provided  below: 

Example  1:  In  traditional  PHQ-2  scoring  (when  a  positive  score  is  a  total  of  three  or 
greater  when  the  responses  to  the  two  questions  are  added),  a  person  who  scored  a  2  on 
the  first  question  and  a  0  on  the  second  question  would  have  a  total  score  of  2,  and  thus 
would  screen  negative.  In  contrast,  according  to  PDHRA  scoring  recommendations,  this 
would  be  a  positive  screen  because  one  question  reached  the  threshold  of  2. 

Example  2:  When  traditional  PHQ-2  additive  scoring  uses  2  as  the  cutoff,  a  person  who 
scores  1  on  both  questions  will  screen  positive  by  that  algorithm,  because  he  will  have  a 
total  score  of  2.  On  the  PDHRA  algorithm,  he  will  screen  negative,  because  he  failed  to 
reach  the  threshold  of  2  on  either  independent  question. 

As  with  the  PTSD  subscale,  there  is  no  place  on  the  provider  portion  of  DD  Form  2900  which 
actually  contains  scoring  instructions,  and  there  is  no  specific  space  for  providers  to  document 
whether  SMs  screened  positive  or  not  (DD  2900  (PDHRA)  -  January  2008  Version,  2008). 

The  literature  review  identified  nine  studies  that  examined  the  validity  of  the  PHQ-2.  Because 
the  PHQ-2,  unlike  the  PTSD  screen,  was  not  originally  a  VA  scale,  a  wide  variety  of  populations 
were  studied  in  the  articles  identified,  including  active  service  military,  general  practice  patients 
in  the  U.S.  and  other  developed  countries,  low-income  and  -education  inner  city  populations, 
coronary  patients,  and  populations  in  developing  nations.  Some  studies  in  this  review  include 
more  than  one  dataset  or  evaluate  against  more  than  one  gold  standard,  so  there  are  actually 
13  datasets  included  (Table  B.2  identifies  the  multiple  datasets  within  studies).  Of  the  studies 
identified,  only  Bliese  et  al.'s  (2005)  3-sample  study  uses  the  scoring  algorithm  recommended 
for  the  PDHRA,  while  eight  other  studies  used  the  original  additive  scoring  method.  Four  of 
these  studies  use  a  previously  validated  structured  interview  as  a  gold  standard,  while  several 
others  used  previously  validated  questionnaires  as  gold  standards  (This  methodology  is  less 
rigorous,  and  so,  while  the  studies  were  retained  in  the  review,  they  are  marked  with  an 
asterisk  in  the  summary  table  at  the  end  of  this  section).  A  summary  of  the  sensitivity  and 
specificity  values  determined  for  several  subsets  of  reviewed  studies  is  included  below,  while 
more  detailed  information  is  included  in  Table  B.2,  which  follows: 
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1.  PDHRA  Algorithm  (1  Study,  3  data  sets):  When  the  PDHRA  algorithm  is  used,  sensitivity 
ranges  from  .50  to  .73  and  specificity  ranges  from  .86  to  .95.  Bliese  et  al.  (2005)  also 
report  on  using  the  same  cutoff  point,  but  requiring  endorsement  of  both  items.  This 
scoring  method  was  only  assessed  for  one  sample,  but  produced  an  unsatisfactory 
sensitivity  of  .31,  and  a  specificity  of  .97.  Therefore,  all  studies  using  this  algorithm 
recommend  a  positive  screen  when  one  item  is  endorsed  rather  than  two. 

2.  Original  PHQ-2  Scoring  Algorithm  with  any  Gold  Standard  (8  Studies,  13  data  sets): 

All  studies  which  used  an  algorithm  requiring  addition  between  the  two  question  scores 
recommend  a  cutoff  of  either  2  or  3.  With  a  cutoff  of  2  studies  reported  sensitivities 
ranging  from  .78  to  1.0  and  specificity  from  .51  to  .89.  With  a  cutoff  of  >3  sensitivity 
ranged  from  .25  to  .97  and  specificity  ranged  from  .78  to  1.0.  (The  .25  figure  was  for  a 
data  subset  of  mothers  with  high  school  education  or  less,  which  the  authors  cited  as 
evidence  that  this  algorithm  with  this  screen  may  not  work  as  well  in  low  education 
populations.  Sensitivity  was  .44  for  the  whole  sample  and  .86  for  more  educated 
mothers). 

3.  Original  PHQ.-2  Scoring  Algorithm  with  Structured  Clinical  Interview  Gold  Standard  (4 
studies,  4  data  sets): 

Not  all  of  the  studies  listed  above  used  a  structured  clinical  interview  as  a  gold  standard. 
Among  the  four  studies  where  structured  clinical  interviews  were  used,  all 
recommended  a  cutoff  of  3  (with  acknowledgement  that  in  some  situations  2  may  be 
better).  With  a  cutoff  of  2,  sensitivity  ranged  from  .90  to  1.0  and  specificity  ranged  from 
.51  to  .74.  With  a  cutoff  of  3,  sensitivity  ranges  from  .74  to  .87  and  specificity  from  .75 
to  .90. 

It  is  encouraging  to  review  the  diversity  of  these  studies  and  to  note  that  acceptable  values  for 
sensitivity  and  specificity  were  found  when  the  screen  was  used  on  a  wide  variety  of  patient 
types,  including  active  service  military,  veterans,  adolescents,  and  OB/GYN  patients  (Bliese,  et 
al.,  2005;  Corson,  et  al.,  2004;  Kroenke,  et  al.,  2003;  Richardson,  et  al.,  2010).  The  scale  was  also 
found  to  be  valid  in  multiple  countries  (Arroll,  et  al.,  2010;  Chen,  et  al.,  2009;  Lowe,  et  al., 

2005),  and  ethnicity  did  not  influence  effectiveness  (Cutler,  et  al.,  2007).  One  study  did  find  that 
the  screen  was  less  sensitive  for  women  with  a  high  school  education  or  less  than  it  was  for 
women  with  some  college,  and  it  is  worth  noting  that  education  may  influence  the  screen's 
effectiveness  (Cutler,  et  al.,  2007).  However,  this  may  not  be  a  major  factor  in  the  military 
population,  in  which  gender  is  predominantly  male  and  all  individuals  have  received  some  form 
of  post-high  school  training  even  if  they  have  not  attended  college. 
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Table  B.2  Summary  of  Sensitivity  and  Specificity  Values  for  the  PHQ-2 


Article 

N 

Study  Population 

Scoring 

Method 

Gold  Standard 

Positive  Score 

Sensitivity 

Specificity 

Arrol  et  al., 
2010 

2642 

Adult  Primary  Care 
Patients,  New 
Zealand 

Original 

PHQ 

*CIDI 

2 

0.86 

0.78 

3 

0.61 

0.92 

Bliese  et 
al.,  2005 

367 

Active  Service 
Post-deployment, 
Army 

PDHRA 

Adapted  MINI 

Either  item 

endorsed 

0.73 

0.86 

Both  items 

endorsed 

0.31 

0.97 

Bliese  et 
al.,  2005 

592 

Active  Service  Post 

PDHRA 

Adapted  MINI 

Either  item 

endorsed 

0.5 

0.95 

Bliese  et 
al.,  2005 

356 

Active  Service  Post 

PDHRA 

Adapted  MINI 

Either  item 

endorsed 

0.65 

0.89 

Chen  et  al., 
2009 

364 

Primary  Care 
Patients  aged  60  or 
over  in  China. 

Original 

PHQ 

SCID 

2 

0.90 

0.66 

3 

0.84 

0.9 

Corson  et 
al.,  2004 

1211 

Veterans  at  VA 
primary  care 
clinics. 

Original 

PHQ 

*PHQscoring  >  10 

as  + 

2 

0.95 

0.89 

3 

0.76 

0.95 

Cutler  et 
al.,  2007 

94 

Low  Income  Urban 

Mothers 

Original 

PHQ 

*PHQ  scoring  >  15 

as  + 

2 

0.99 

0.79 

3 

0.93 

0.86 

*The  Major 
Depression  Algorithm 

2 

1.00 

0.81 

3 

0.97 

0.91 

*Edinburgh 
Postnatal 
Depression  Scale 
(EPDS) 

2  (All 

Participants) 

0.78 

0.87 

3  (All 

Participants) 

0.44 

0.97 

Kroenke  et 
al.,  2003 

580 

Adult  primary  care 
patients  and  adult 
OB/GYN  patients 

Original 

PHQ 

Structured  clinical 
interview  adapted 
from  SCID  and 

PRIME-MD 

3  (Mother 
<HS Ed) 

0.25 

0.95 

3  (Mother  > 
Some  College) 

0.86 

1.00 

2 

.93 

.74 

3 

.83 

.90 

Lowe  et 
al.,  2005 

520 

German 
Outpatients  at 
outpatient  clinics 
and  family 
practices 

Original 

PHQ 

SCID 

2 

1.00 

0.51 

3 

0.87 

0.78 

Richardson 
et  al.,  2010 

444 

Adolescents 

Original 

PHQ 

*PHQ-9 

2 

1.00 

0.62 

3 

0.96 

0.82 

Thombs  et 
al.,  2008 

1024 

Outpatients  with 
coronary  artery 

disease 

Original 

PHQ 

DISC-IV  Interview 

2 

0.9 

0.57 

3 

0.74 

0.75 

*C-DIS 

2 

0.82 

0.79 

3 

0.39 

0.93 

*Denotes  a  gold  standard  that  is  not  a  structured  clinical  interview. 

Note:  Bold  in  the  Algorithm  column  denotes  the  algorithm  recommended  by  the  study's  authors. 
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The  Relationship  Problems  Subscale 

The  relationship  problems  screen  in  the  PDHRA  consists  of  a  single  item  in  which  SMs  are  asked 
to  answer  "yes,"  "no,"  or  "unsure"  to  the  question  "since  return  from  your  deployment,  have 
you  had  serious  conflicts  with  your  spouse,  family  members,  close  friends,  or  at  work  that 
continue  to  cause  you  worry  or  concern?"  A  careful  review  of  the  literature  revealed  that 
neither  this  screen  nor  any  similar  single  item  screen  for  relationship  problems  has  been 
previously  validated  in  a  peer-reviewed  publication.  However,  one  non-peer  reviewed  report 
did  assess  validity  of  the  PDHRA  item  for  a  post-deployment  sample  of  active  service  army 
personnel  (Bliese,  et  al.,  2005).  The  main  question  to  be  addressed  when  scoring  the 
relationship  question  is  whether  to  interpret  answers  of  "unsure"  as  positive  or  negative 
screens.  Bliese  et  al.  (2005)  determined  sensitivity  and  specificity  values  for  both  ways  of 
scoring  this  item.  They  found  that  when  "unsure"  was  scored  as  "no,"  sensitivity  was  .58  and 
specificity  was  .88.  When  "unsure"  was  scored  as  "yes,"  sensitivity  was  .68  and  specificity  was 
.81.  Sensitivity  was  lower  among  married  SMs,  suggesting  that  the  screen  may  miss  more 
relationship  problems  in  this  group.  However,  for  both  groups,  the  best  balance  of  sensitivity 
and  specificity  could  be  achieved  by  scoring  "unsure"  responses  as  positive  (Bliese,  et  al.,  2005). 
This  is  the  algorithm  currently  recommended  for  scoring  the  PDHRA  (Wright,  et  al.,  2008).  As 
with  the  two  previously  reviewed  subscales,  the  PDHRA  form  does  not  include  these 
instructions,  nor  does  it  include  a  place  for  providers  to  record  the  score  on  the  provider 
documentation  page  (DD  2900  (PDHRA)  -  January  2008  Version,  2008). 

The  Traumatic  Brain  Injury  (TBI)  Subscale 

The  PDHRA  TBI  scale  is  identical  in  content  to  a  four-item  TBI  screening  instrument  that  was 
implemented  in  the  VA  system  in  2007.  This  scale,  in  turn,  is  based  on  the  three  item  Brief 
Traumatic  Brain  Injury  Screen  (BTIS)  developed  specifically  for  active  duty  military  personnel 
and  validated  by  the  Defense  and  Veterans  brain  Injury  Center  (Carlson,  et  al.,  2010;  GAO, 

2008)  The  VA  TBI  screen  roughly  corresponds  to  the  BTIS  screen,  but  with  the  addition  of  one 
item;  while  the  BTIS  only  asks  about  symptoms  that  occurred  immediately  after  the  injury  and 
at  the  time  of  the  screening,  the  VA  scale  asks  about  these  time  points  in  addition  to  asking 
about  problems  that  began  or  got  worse  after  the  injury  event.  The  wording  of  the  three 
VA/PDHRA  instrument  items  that  correspond  to  the  BTIS  items  is  similar,  but  not  identical  to, 
the  BTIS. 

In  VA  settings,  a  positive  screen  on  the  four  item  instrument  is  coded  when  the  veteran 
endorses  at  least  one  item  within  each  of  four  sections.  Veterans  with  affirmative  responses  to 
the  first  two  sections  but  negative  responses  to  the  third  and  fourth  sections  are  not 
systematically  referred  for  a  follow-up  evaluation  for  TBI,  since  post-deployment  screening  aims 
to  find  continuing  problems  rather  than  problems  that  resolved  earlier  (Carlson,  et  al.,  2010). 
Although  the  PDHRA  is  administered  to  active  duty  SMs,  the  context  is  similar  to  that  of  the  VA 
in  that  SMs  reporting  for  screening  will  almost  certainly  be  at  least  three  months  removed  from 
any  deployment-related  head  injury  events.  Recommended  PDHRA  scoring  is  similar  to  that 
which  is  recommended  in  the  VA  setting,  with  instructional  notes  on  the  provider  portion  of  the 
form  indicating  that  a  positive  screen  for  potential  TBI  with  persistent  symptoms  should  be 
coded  based  on  responses  to  the  last  item,  which  asks  SMs  to  report  symptoms  experienced 
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within  the  past  week.  The  provider  form  also  includes  a  space  to  specifically  record  the  results 
for  this  screen,  in  contrast  to  the  lack  of  documentation  expected  for  the  PTSD,  depression,  and 
relationship  screens  ( DD  2900  (PDHRA)  -  January  2008  Version,  2008). 

The  literature  contains  no  study  validating  this  scale,  either  as  it  is  applied  in  the  PDHRA  process 
or  in  its  use  as  a  VA  screening  tool.  The  scale's  validity  in  the  VA  system  is  currently  being 
investigated,  through  no  results  of  this  study  can  be  expected  before  December  2011  (Babcock- 
Parziale,  2009).  Some  validation  of  the  BTBIS  screen  has  been  conducted,  though  it  has  not 
been  extensive.  No  sensitivity  or  specificity  data  are  available  for  BTBIS,  and  the  only  indicator 
of  validity  available  in  the  published  literature  is  a  study  by  Schwab  et  al.  (2007),  which  found 
that  the  prevalence  of  having  three  or  more  postconcussive  syndrome  symptoms  was  higher  (at 
74%)  among  SMs  who  reported  recent  symptoms  on  the  third  BTBIS  question  than  it  was  for 
those  who  did  not  report  problems  (at  40%).  This  lack  of  validation  for  TBI  screening 
instruments  in  the  military  has  led  to  concern  about  misdiagnosis  among  some  military 
providers  and  researchers  (Hoge,  et  al.,  2009). 

The  Alcohol  Use  Subscale 

The  alcohol  screen  used  on  the  PDHRA  consists  of  one  5-part  question  (ql3a-e),  of  which  the 
first  two  items  are  derived  from  the  Two  Item  Conjoint  Screen  for  Alcohol  and  Other  Drug 
Problems  (TICS)  (Brown,  Leonard,  Saunders,  &  Papasouliotis,  2001),  and  the  last  three  are  the 
questions  that  make  up  the  AUDIT-C  (Bush,  Kivlahan,  McDonell,  Fihn,  &  Bradley,  1998).  The 
PDHRA  form  of  the  AUDIT-C  is  consistent  with  other  versions  of  the  instrument  in  common  use, 
but  there  are  two  important  differences  between  Brown  et  al.'s  (2001)  original  TICS  scale  and 
the  PDHRA  version.  First,  questions  in  the  TICS  scale  ask  respondents  about  their  behaviors  over 
the  course  of  the  past  year,  while  the  PDHRA  asks  about  the  past  month,  a  change  that  was 
made  to  account  for  the  fact  that  SM  drinking  patterns  usually  differed  from  the  present  during 
deployment  (Wright,  et  al.,  2008).  The  second  way  in  which  the  PDHRA  screen  was  changed 
from  the  original  is  that  both  questions  were  altered  to  ask  only  about  alcohol  as  opposed  to 
encompassing  both  alcohol  and  drug  use. 

Each  of  these  instruments  has  been  validated  independently,  though  no  validation  study  has 
examined  the  use  of  the  two  scales  in  combination.  Brown  et  al.  (2001)  found  sensitivity  and 
specificity  values  of  .70  and  .78,  respectively  for  the  TICS,  noting  that  because  the  instrument 
asked  about  drug  or  alcohol  use  without  requiring  the  respondent  to  specify  which  one  they 
consumed,  it  was  possible  that  sensitivity  might  be  lowered  for  detection  of  alcohol  problems, 
if  problem  drinkers  feared  that  a  positive  response  would  cause  people  to  believe  that  they 
used  illegal  drugs  (Brown,  et  al.,  2001).  The  questions  used  on  the  PDHRA  make  reference  to 
alcohol  only,  so  if  this  were  indeed  a  factor,  the  effects  might  be  mitigated  by  this  approach; 
however,  no  literature  identified  in  this  review  validated  the  modified  TICS  used  on  the  PDHRA. 

The  AUDIT-C  has  been  validated  in  multiple  studies  conducted  both  in  VA  and  in  other 
populations,  with  sensitivity  scores  ranging  from  .81  to  .95  and  specificity  ranging  from. 69  to 
.86  for  population  samples  including  both  men  and  women  (Bradley,  et  al.,  2003;  Bush,  et  al., 
1998;  Gordon,  et  al.,  2001).  At  least  one  study  has  shown  that  using  a  cutoff  point  of  3  for 
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women,  as  is  standard  in  PDHRA  scoring  is  necessary  to  achieve  acceptable  validity  of  the 
AUDIT-C  for  women  (Dawson,  Grant,  Stinson,  &  Zhou,  2005).  There  is  also  evidence  that  AUDIT- 
C  scales  supplemented  by  one  or  two  additional  questions  may  be  more  valid  (Bradley,  et  al., 
2004;  Kriston,  et  al.,  2008),  though  it  is  impossible  to  say  whether  this  is  true  of  the 
combination  of  AUDIT-C  and  TICS.  Further,  in  previous  studies  that  examined  slightly  expanded 
AUDIT-C  instruments,  the  new  questions  were  incorporated  into  the  AUDIT-C  scoring  algorithm. 
On  the  PDHRA,  the  two  scales  are  scored  separately,  and  a  positive  screen  is  indicated  when  an 
SM  has  a  total  score  of  4  (3  for  women)  on  the  AUDIT-C  component  (13c-e)  or  an  endorsement 
of  "yes"  for  either  question  13a  or  13b.  These  directions  are  specifically  provided  to  the 
interviewer  on  the  form,  along  with  a  space  where  the  result  of  the  screen  is  specifically 
recorded  ( DD  2900  (PDHRA)  -  January  2008  Version,  2008). 

Comorbidity  and  Cross-Scale  Screening  Issues 

The  review  above  shows  that  the  PDHRA  form  contains  only  limited  guidance  on  whether  to 
favor  sensitivity  or  specificity  when  interpreting  individual  scales.  However,  no  guidance  at  all  is 
included  to  help  providers  interpret  the  results  of  multiple  scales  in  relation  to  one  another. 
Some  have  argued  that  favoring  specificity  may  not  reduce  recognition  of  comorbidities,  an 
assertion  based  on  the  idea  that  as  long  as  an  SM  screens  positive  on  one  scale,  the  provider 
will  be  alerted  to  look  for  additional  problems  during  the  interview  (Bliese,  et  al.,  2005).  Others 
have  provided  evidence  to  the  contrary.  For  example,  in  one  VA  clinic,  25%  of  substance  abuse 
disorder  patients  were  found  to  have  previously  undiagnosed  PTSD  in  spite  of  the  fact  that  they 
had  already  been  screened  and  interviewed  for  mental  health  concerns  (Kimerling,  et  al.,  2006). 
High  rates  of  previously  unidentified  comorbidities  have  also  been  reported  among  patients 
diagnosed  with  TBI  (Carlson,  et  al.,  2010).  These  findings  indicate  that  clinical  interviews  do  not 
always  succeed  in  identifying  multiple  mental  health  concerns.  Further,  comorbidities  continue 
to  be  missed  even  after  patients  have  entered  treatment  for  other  problems.  PDHRA  providers 
should  not  make  the  mistake  of  assuming  that  as  long  as  they  refer  a  patient  for  one  condition, 
other  providers  will  identify  any  problems  that  they  missed.  Effective  feedback  that  increases 
the  accuracy  of  the  PDHRA  interview  has  the  potential  to  increase  the  chances  that  providers 
will  make  appropriate  referrals  and  that  SMs  will  be  able  to  access  appropriate  care. 
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APPENDIX  C:  STUDY  DESIGN  PRESENTATION  TO  FHP&R  AND  DCOE 
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Methods 

-  Purdue  Subcontract 

•  Study  Timeline 

•  Potential  Modification  to  Study  Design  and 
Timeline 
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Previous  Research 

•  NOV  2007  -  DEC  2009,  VU  conducted  a  program 
evaluation  to  assess  how  the  PDHA  and  PDHRA 
contribute  to  increasing  Service  Member  (SM) 
access  to  appropriate  care 

•  CEPI  background  in  program  evaluation  research 
and  specifically,  training  and  feedback 
interventions  to  change  clinician  behavior 


Previous  Research:  Evaluation 

Design 

•  Multiple  methods  and  data  sources 

-  PDHA,  PDHRA,  and  health  care  records 

-  SM  survey 

-  Quasi-experimental  study  of  effect  of  SM 
education 

-  Qualitative  interviews  with  key  stakeholders  in 
PDHRA  process 

-  Observations  at  10  different  installations 

-  Content  analysis  of  audio-recorded  PDHRA 

interviews  4 
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Previous  Research:  Key  Relevant 

Findings 

•  Clinician  interview  adds  value  to  increasing  medical  referrals, 
but  this  contribution  is  relatively  small  compared  to  the 
contribution  of  SM  self-reported  problems 

•  Clinical  interview  focuses  on  SM  self-reported  problems  and 
misses  problems  for  SMs  who  do  not  already  disclose  them  on 
PDHRA  self-report.  The  interview  does  little  to  increase 
sensitivity 

•  Significant  minority  of  SMs  admit  to  underreporting  behavioral 
health  problems  on  the  PDHRA 

•  Low  reliability  in  clinician  interviews  across  multiple  PDHRAs  for 
same  deployment,  despite  high  consistency  for  SMs’  self-report 

•  PDHRA-specific  communication  about  behavioral  health  issues 
occurs  less  than  for  physical  issues 

•  Clinicians  do  not  use  empathy  statements 

•  Reported  lack  of  systematic  and  intensive  clinician  training  5 
specific  to  PDHRA 


Key  Relevant  Recommendations 
From  Previous  Research 

•  Establish  quality  assurance  procedures  for  the  clinician  interview 

-  Ensure  providers  ask  about  all  potential  health  problems,  even  those  not 
endorsed,  and  especially  behavior  health  issues 

-  Providing  education 

•  Establish  clinician  guidelines  for  a  more  structured  and  systematic 
PDHRA  interview 

-  Increase  reliability  of  clinician  screening 

•  Require  all  interviewers  to  have  successfully  completed  PDHRA- 
specific  training 

-  Increased  normalization  of  behavioral  health  issues  upon  return  from 
deployment 

-  How  to  minimize  under-reporting 

-  How  to  deal  with  the  alcohol  questions 

•  Provide  clinicians  with  monitoring  and  feedback  about  their 
performance  on  the  PDHRA  to  ensure  ongoing  quality  improvement 
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Current  Proposal 

•  Funded  by  US  Army  Medical  Research 
Acquisition  Activity  (USAMRAA) 

•  Government  sponsor  is  FHP&R,  with  LCDR 
Nicole  Frazer  and  Dr.  Mark  Paris  as  co¬ 
investigators 

•  SEP  2009  -  SEP  201 1  with  option  for  one 
year  extension 


Purpose 

•  To  develop  and  test  the  effectiveness  of  a 
targeted  training  and  feedback  intervention 
designed  to  help  providers  increase  SM 
reports  of  behavioral  health  concerns  and 
SM  acceptance  of  a  referral  for  further 
assessment  where  warranted. 
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Study  Design 

•  Aim  1 :  Determine  key  elements  of  and  current 
impact  of  training  programs  for  deployment 
related  assessments 

-  Expert  Panel  review  of  2007-2009  collaboration 
between  VU  and  FHP&R  to  determine  criteria  for 
clinical  competency 

-  Focus  groups  with  key  stakeholders  involved  in  the 
PDHRA  process 

-  Secondary  analysis  of  PDHRA  data  from  a  specifically 
developed  database  that  includes  provider  and  MTF 
identifiers 


Study  Design 


•  Aim  2:  Evaluate  the  effectiveness  of  a  targeted  training  and  feedback 
program  on  primary  care  providers’  interview  and  clinical 
communication  patterns  related  to  SM  behavioral  health  condition 
identification  and  referrals 

-  Based  on  findings  from  Aim  1 

-  Clinicians  randomly  assigned  to  one  of  two  experimental  groups  (1 ) 
training  and  ongoing  feedback,  (2)  training  only,  or  to  a  control  group  to 
receive  whatever  training  they  would  typically  receive 

-  39  PDHRA  clinicians  at  4-6  MTFs 

•  Implementation  will  be  measured  through  training  attendance  records, 
evaluations  completed  by  the  providers,  and  review  of  taped  simulated 
interviews  as  a  formative  assessment 

•  Outcome  measurement  will  include  pre-and  post-training  content 
analysis  of  communication  style  from  audiotapes  and  an  analysis  of 
secondary  data 

10 
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Risk  Factors  Associated  with  the 
Development  of  PTSD  -  Purdue 
Subcontract 

•  Primary  focus  -  identify  risk  and  resiliency  factors 
associated  with  the  development  of  PTSD. 

•  Secondary  focus  -  assess  the  correspondence 
between  initial  self-report  and  primary-care 
provider  assessments  of  trauma-related 
symptomatology  on  the  PDHA  and  PDHRA  and 
subsequent  diagnosis  for  PTSD. 


Potential  Risk  Factors 


Individual-level  factors 

-  Marital/family  status 

-  Age 

-  Pre-existing  medical/mental 
health  conditions 

Service-related  factors 

-  Branch 

-  Rank 

-  Time  in  service 

-  Pay  grade 


Deployment-specific 

factors 

-  Number  of  deployments 

-  Total  time  deployed 

-  Length  of  time  for  each 
deployment 

-  Dwell  time  between 
deployments 

-  Frequency/intensity  of 
combat  exposure 

12 
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Identifying  Risk  and  Resiliency 
Factors  Associated  with  PTSD 

•  Analysis  of  records  from  AFHSC  (subset  of  VU  data  set) 
and  VHA 

•  Observational,  retrospective  case-control  design 

•  "Cases"  will  be  defined  as  SMs  who,  after  first 
deployment,  met  the  current  criteria  for  confirmed  PTSD 
(at  least  one  in-patient  encounter  or  two  outpatient 
encounters  with  a  diagnostic  code  indicating  PTSD 
(ICD=309.81))  between  March  2005  and  December  2008. 

•  "Controls"  will  be  selected  randomly  and  will  be  matched 
to  cases  on  gender,  service  component,  military 
occupational  specialty,  and  will  have  been  deployed  in  the 
6  mo  prior  to  the  case’s  PTSD  diagnosis 
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Concordance  Study 

•  Concordance  between  SM  self-reported 
problems,  provider-reported  concerns,  and 
referral  characteristics 

•  Concordance  between  PDHRA  and 
subsequent  health  care  utilization  (including 
VHA  utilization) 
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Study  Timeline 

•  First  year  (SEP  2009 -SEP  2010) 

-  Secondary  analysis  of  PDHRA  data  including  provider  and  MTF 
identifiers  and  review  of  findings  from  previous  collaboration 

-  Focus  groups 

-  Develop  feedback  and  training  intervention 

-  Site  recruitment 

•  Second  year  (SEP  2010 -SEP  2011) 

-  Implement  the  training  and  feedback  program  and  evaluate  the 
effectiveness  and  feasibility  of  the  intervention 

•  Outcomes  include  implementation  fidelity  and  quality, 
content  analysis  of  communication  style  from  interview 
audiotapes,  secondary  analysis  of  the  PDHRA  form  and 
SM  health  care  utilization,  and  SM  satisfaction  surveys. 

15 


Recent  NDAA  Legislation 


•  HR  2647  Sec  708 

-  Requires  person-to-person  mental  health  assessments  for  SMs 
deployed  with  a  contingency  operation 

-  Assessments  will  occur  60  days  pre-deployment,  and  at  3-6  mo, 
12  mo,  and  24  months  post-deployment 

-  Clinicians  must  be  trained  and  certified  to  conduct  the 
assessments 

•  Timeline 

-  26  APR  2010  -  Guidance  to  be  issued  to  the  Services 

-  21  JAN  201 1  -  Initial  report  to  Congress  on  implementation 

-  26  APR  2012  -  report  to  Congress  on  implementation,  including 
“an  evidence-based  assessment  of  the  effectiveness  of  the  mental 
health  assessments...”  (HR  2647,  pg  189) 
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Proposed  Modifications  to  Study 

Design 

•  Prioritize  development  of  a  clinician  training  program  to 

inform  the  guidance  consistent  with  implementation  of 

NDAA  legislation 

•  Now  -  April  2010:  Using  evidence  to  inform  guidance 

-  Review  of  existing  programs  and  methods  by  content  experts 
(behavioral  health),  context  experts  (military),  and  program 
evaluation/feedback  experts  (VU) 

-  Gather  data  on  existing  identified  ‘best  practice’  training  and/or 
monitoring  programs  for  PDHRA  clinicians 

-  Further  develop  concrete  recommendations  from  existing  data 
relevant  to  behavioral  health  assessment  and  clinician  training 

-  Recommend  guidance  include  stipulations  for  (1)  a  randomized 
controlled  study  and  (2)  training  and  ongoing  feedback  model 

-  Identify  expert  panel  and  key  stakeholders  17 


Proposed  Modifications  to  Study 

Design 

•  April  -  October  2010:  Development 

-  Training  materials  and  protocols 

-  Evaluation  design 

-  Approvals  process 

•  October  201 0  -  January  2011:  Formative 
evaluation  and  report 

•  January  201 1  -  April  201 2:  Summative  evaluation 
and  report 
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APPENDIX  D:  RESEARCH  AND  INTERVENTION  DESIGN  PRESENTATION 

TO  EXPERT  PANEL 
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Improving  Deployment-Related 
Primary  Care  Provider 
Assessments  of  Behavioral 
Health  Conditions 

Susan  Kelley,  Ph.D. 

Len  Bickman,  Ph.D. 

Melanie  Leslie,  Ph.D. 

Center  for  Evaluation  and  Program  Improvement 
Vanderbilt  University 
July  21, 2010 


Presentation  Goals 

•  Establishing  the  need  for  provider  training 

-  Describe  current  study  partnership  and  objectives 

-  Review  purpose  and  goals  of  PDHRA  interview 

-  Review  key  findings  of  previous  PDHRA  evaluation 

•  Importance  of  feedback  in  improving  performance 

•  Proposed  Training  and  Feedback  Program: 

3  components 

1 .  Initial  training 

2.  Decision  support  during  interview 

3.  Aggregated  feedback  on  a  weekly  basis 

•  Engage  Expert  Panel  guidance  and  determine  next 
steps  for  development 
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Current  Study 

•  VU/FHP&R  partnership 

•  Cooperative  agreement  funded  by  US  Army  Medical  Research 
Acquisition  Activity  (USAMRAA) 

•  Purpose:  Develop  and  pilot  a  demonstration  of  a  targeted  training 
and  feedback  program  to  enhance  health  care  providers’  clinical 
communication  practices  and  identification  of  SM  behavioral  health 
problems  warranting  further  evaluation 

-  Focus:  PDHRA  interview 

-  Problem  areas:  PTSD,  depression,  relationship  conflict,  alcohol  use,  TBI 

-  Objectives 

1.  Analyze  data  using  predictive  modeling  to  develop  feedback  elements 

2.  Implement  pilot  program  and  study  outcomes  related  to  feasibility  and 
effectiveness 

•  Timeline:  SEP  2009  -  SEP  201 1  with  option  for  one  year  extension 
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Goals  of  the  PDHRA* 

•  PDHRA  purpose:  Population-based  screening  for 
deployment-related  physical  and  behavioral  health 
concerns 

-  To  increase  SM  access  to  appropriate  care  for  SMs  in  need  of 
evaluation 

-  To  document  deployment-related  concerns  in  all  SMs’  medical 
records  (PDHRA  encounter  coded  as  V70.5) 

•  Specific  goals  of  the  PDHRA  provider  interview 

-  Clarify  and  confirm  SM  responses  on  the  DD  Form  2900 

-  Educate  SMs  about  concerns,  healthcare,  and  treatment  options 

-  Conduct  a  risk  assessment 

-  Make  referrals  for  further  evaluation  where  warranted 

'According  to  HA  Policy  05-01 1  (10  Mar  05)  and  current  information  available  on  www.pdhealth.mil 
(e.g.,  http://www.Ddhealth.mil/dcs/downloads/PDHRABackaround  Paper4  9Jan06  03  22  06,pdn  4 
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Our  Previous  PDHRA  Evaluation 

•  Collaboration  with  FHP&R  to  conduct  a  two-year 
evaluation  of  the  PDHA/PDHRA  process  encompassing 
all  Branches  and  Components  of  the  military  (Sep  07  - 
Dec  09) 

•  This  project  evaluated  if  the  PDHRA  process  increased 
SM  access  to  appropriate  care 

-  Components:  self-report  ->  provider  interview  -> 
health  care  utilization 

-  Process 

•  Relationship  to  PDHA 

•  Characteristics  of  setting,  interview,  SMs 


Previous  PDHRA  Evaluation: 
Methods 

•  Multiple  methods  and  data  sources 

•  Major  components 

-  Analysis  of  PDHA  (n=300k),  PDHRA  (n=251k),  and 
health  care  utilization  (n=2.1M)  for  SMs  between  01- 
JAN-06  and  31-MAR-09.  Included  all  Branches  and 
Components 

-  SM  survey  (n=6,7 1 4)  to  identify  SM-related  factors 
that  may  influence  the  PDHRA  process  and 
satisfaction  with  the  process 

-  Quasi-experimental  study  (n=766)  to  examine  the 
effects  of  Battlemind  II  on  SM  reporting  and  referral 
acceptance 
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Previous  PDHRA  Evaluation: 
Methods  (cont) 

•  Major  components  (cont) 

-  Semi-structured  interviews  with  100  key  participants 
in  the  PDHRA  process  and  observations  of  PDHRA 
process  at  10  installations 

-  Content  analysis  of  audio  recordings  of  272  PDHRA 
provider  interview  calls  (Reserve  component  only) 

-  Analysis  of  PDHRA  context  (e.g.,  in-person  v. 
telephone  interview)  using  52,556  records  provided 
by  contracting  agency  for  PDHRAS  completed 
between  February  2008  and  March  2009  (Reserve 
component  only) 


Key  Evaluation  Findings: 

SM  factors 

•  SM  PDHRA  self-report:  PDHA  provides 
valuable  information 

-  PDHA  responses  strongly  predict  PDHRA 
responses 

•  Especially  for  TBI  (OR=21.6),  PTSD  (OR=7.7),  and 
depressive  symptoms  (OR=8.0) 

•  Intervening  health  care  encounters  did  not  influence 
PDHRA  reporting 

-  SMs  who  reported  combat  exposure  on  PDHA 
twice  as  likely  to  endorse  problems  on  PDHRA 
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Key  Evaluation  Findings: 

SM  factors 

•  SM  Survey:  Barriers  to  reporting  behavioral  health 
problems 

-  Significant  minority  (10-14%)  of  SMs  admit  to 
underreporting  physical,  emotional,  or  alcohol  use 
problems  on  the  PDHRA 

-  34%  of  SMs  anonymously  reported  experiencing 
behavioral  health  problems  since  deployment 

•  Negative  attitudes  toward  help-seeking  and  accepting  mental 
health  treatment 

•  Almost  half  (43%)  reported  no  such  problems  on  PDHRA 


Key  Research  Findings: 
Provider  Factors 

•  The  interview  does  little  to  increase  sensitivity,  focusing 
on  SM  self-reported  problems  and  missing  non-disclosed 
problems. 

-  Providers  reported  using  positive  responses  from  SM  self-report 
to  guide  interviews 

-  Number  of  major  concerns  and  presence  of  a  medical  referral 
was  compared  for  two  groups  of  SMs:  (1 )  Non-  Disclosers  - 
reported  behavioral  health  problems  on  SM  survey,  but  not  on  PDHRA 
(n=282);  (2)  Disclosers  -  reported  behavioral  health  problems  on  SM 
survey  and  on  PDHRA  (n=380) 

•  Average  number  of  provider  major  concerns  five  times  lower  for 
non-disclosers  (mean  =  0.11)  compared  to  disclosers  (mean  =  0.53) 

•  Receipt  of  at  least  one  medical  referral  almost  three  times  lower  for 
non-disclosers  (mean  =  0.14)  compared  to  disclosers  (mean  =  0.67) 
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Key  Research  Findings: 
Provider  Factors 

•  PDHRA-specific  communication  about  behavioral  health  issues 
occurs  less  than  for  physical  issues 

1 .  Less  mention  of  behavioral  health  problem  areas 

•  Physical  health  mentioned  regardless  of  SM  endorsement  (87%  v  84%) 

•  Behavioral  topics  mentioned  more  when  SM  endorsed  (64%)  than  not 
endorsed  (35%) 

2.  Little  provision  of  education 

•  Education  related  to  mental  health  issues  in  14%  of  all  calls 

•  Increased  to  24%  in  calls  where  a  medical  referral  was  given 

3.  Prior  treatment  discussed  less  for  behavioral  health  and 
exposure  problems 

•  Physical  health  or  TBI  problems  (87%  to  92%) 

•  Alcohol,  mental  health,  or  exposure  (3.1%  to  63.6%) 

•  Lack  of  use  of  best  provider  communication  strategies  to  elicit  more 
self-disclosure  (e.g.,  empathic  statements) 
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Behavioral  Health  Problems 
are  Common 


•  PDHRA  self-report  by  SMs 

-  24%  one  or  more  symptoms  of  PTSD 

-  1 0%  one  or  more  symptoms  of  depression 

-  13%  single  item  on  relationship  conflict 

-  Rates  are  similar  to  SM  reports  of  other  types  of  problems 

•  29%  one  or  more  physical  health  symptoms 

•  25%  one  or  more  exposure  concerns 

•  14%  one  or  more  TBI  symptoms 


According  to  published  studies,  the  presence  of  PTSD 
symptoms  or  diagnosis  in  OIF/OEF  veterans  ranges 
from  5-20%  depending  on  the  measure  and  criteria  used 

for  evaluation  (Hoge  et  a!.,  2004;  Hoge  et  a!.,  2006;  Seal  et  a!.,  2007;  Seal  et  a!.,  2009;  Milliken  et  ai.  2007; 


Tanelian  &  Jaycox,  2008) 
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Recent  News  Stories  Highlight  Concerns  about 
TBI  and  Behavioral  Health  Problems 

•  TBI  is  the  issue  in  a  recent  (Jun  9,  2010)  news  story  by 
NPR  and  ProPublica,  Top  Officer  Says  Military  Takes 
Brain  Injuries  ‘Extremely  Seriously’ 1 

-  “Official  military  statistics  say  115,000  troops  have  suffered 
a  mild  traumatic  brain  injury  since  the  wars  began.  But  in 
interviews,  top  Army  medical  officials  acknowledged  that 
those  figures  understate  the  true  number.” 

•  A  recent  PressTV  article  (May  18,  2010)  is  titled  ‘Mental 
Disorders  Climb  in  US  Military’ 2 

-  “For  the  first  time,  more  US  soldiers  are  hospitalized  for 
serious  mental  disorders  from  their  military  service  than  for 
injuries  and  battlefield  wounds,  according  to  new  medical 
data  released  by  the  Pentagon.  In  2009,  there  were 
17,538  US  soldiers  put  into  hospitals  for  mental  health 
problems  compared  to  17,354  for  battle  wounds  and 
injuries  sustained  during  military  service.” 

1  http://www.propublica.ora/article/top-officer-savs-militarv-takes-brain-iniuries-extremelv-seriouslv  14 

2http  ://www.Dresstv.ir/Dop/Print/?id=1 26861 


Recommendations  for  Providers 

•  Provide  individualized  information  on  risk  factors 
from  PDHA 

•  Improve  consistency  of  interview  and 
documentation 

•  Increase  communication  about  behavioral  health 

•  Use  best  strategies  to  increase  self-disclosure 

Accomplished  Through  Feedback  and  Training 
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Our  Previous  Experience 
Developing  and  Implementing 
Feedback  Systems 

Contextualized  Feedback  System  (CFS):  A 
Systematic  Monitoring  and  Feedback 
Practice  Improvement  Tool 


Strong  Support  for  Feedback 
Concept 

•  Practice  without  feedback  does  not  lead  to 
improvement 

•  Measurement  and  feedback  are  at  the  core  of  all 
management  and  learning  theories 

•  Thousands  of  studies  outside  of  health/mental 
health  show  that  improvement  is  minimal  without 
measuring  performance  and  providing  feedback 

•  Decision  support  and  feedback  show  promise  in 

changing  provider  behavior,  increasing  access 
to  specialized  care,  and  improving  outcomes  in 
health/mental  health  care  17 
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Improving  Performance  Through 

Feedback 

•  Basis  in  continuous  quality  improvement  (CQI) 

•  Feedback  cycle 

-  GENERAL:  input  ->  decision  ->  action  ->  was  that 
action  correct? 

-  PDHRA:  self-report  &  interaction  ->  need  for  further 
evaluation?  ->  referral  ->  was  referral  warranted? 

•  Effective  feedback  is 

-  Timely  and  frequent 

-  Targeted  to  purpose  of  action 

-  Actionable 

-  Easy  to  use  and  integrated  into  clinical  practice 

•  Ambiguous  input  -»  variability  in  interpretation 
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CFS  in  Use 

•  In  mental  health  care 

-  Initial  development  and  testing  (NIMH  R01  8/04-4/10) 

•  Treatment  as  usual  in  over  38  sites  across  the  country  with  over  300 
providers 

•  Evidence  of  effectiveness  in  changing  provider  behavior  and  client  outcomes 

-  Revised  software  and  support  package  (AHRQ  7/09-6/12) 

•  Focus  on  feasibility  in  state  mental  health  system 

•  Increased  emphasis  on  pre-  and  post-implementation  support 

-  Paired  with  evidence-based  treatment  (NIMH  R01  7/10-6/15) 

•  Increased  emphasis  on  decision  support 

•  Testing  increased  effectiveness  of  feedback  on  evidence-based  treatment 

•  In  education 

-  Initial  development  and  testing  (IES  3/07-2/1 1 ) 

•  Teacher  feedback  to  improve  principal  performance 

•  CFS  implemented  at  72  schools,  over  2000  teachers 

-  Grant  proposal  for  replication  19 
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CFS  General  Development 
Process 

•  Basis  in  theory  (individual  and  organizational)  and 
infrastructure  (software) 

•  Collaborate  with  setting  and  content  experts  to 
determine  what  and  how  to  feedback  to  providers 

•  Comprehensive  support  for  implementation  to  promote 
integration  of  feedback 

•  Attention  to  actionable  feedback  through  provision  of 
training  and  coaching 
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Proposed 

Training  and  Feedback 
Program  for  PDHRA  Providers 
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Challenge:  PDHRA  is  Unique 
Medical  Encounter 

•  Brief  (typically  <  1 0  minutes) 

•  One-time  encounter  designed  to  screen,  not  diagnose 

•  Variability  in  site-based  resources  and  rules  for  referral  (formal  or 
informal) 

•  PDHRA  isolated  from  other  relevant  information  (PDHA,  deployment 
history) 

•  SM  participation  is  required 

•  Stigma  of  reporting  behavioral  health  concerns 

•  Limited  and  untested  PDHRA-specific  training  (see-observed-do); 
variability  among  providers 

•  Lack  of  guidelines  for  interpreting  self-report  data  (exception  of 
AUDIT,  TBI) 

•  HCPs  may  have  general  skills  and  training,  but  are  being  asked  to 
conduct  a  specific  type  of  screening 
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Program  Purpose 

•  Overall  goals 

-  Improve  consistency  of  provider  evaluation  (i.e.,  provide  guidelines  for 
interpreting  the  PDHRA  self-report) 

-  Enhance  sensitivity  to  behavioral  health  issues 

•  Our  program  will... 

-  Work  within  the  current  process  without  substantially  altering  the 
purpose  or  time  frame  of  the  interview 

-  Allow  providers  to  focus  on  the  SM  rather  than  interpreting  the  self- 
report 

-  Adapt  according  to  implementation  of  NDAA  Sec.  708 

•  Our  program  will  not... 

-  Follow-up  on  individual  SMs  to  evaluate  the  appropriateness  of  referrals 
or  effectiveness  of  care.  Ultimately,  looking  at  effectiveness  of  care 
would  be  ideal,  but  it  is  outside  the  scope  of  this  study. 
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Three  Components 

•  Overview 

1 .  Brief  training  specific  to  PDHRA 

2.  Decision  support  for  interpretation  of  SM  self-report 
and  other  relevant  information 

3.  Feedback  to  monitor  performance  and  target  areas 
for  improvement 

•  Tailored  to  PDHRA  process  to  enhance 
feasibility 

-  Brief 

-  Designed  for  one-time  contact 

-  Incorporated  into  interviews  and  weekly  staffings 
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Component  1:  Initial  Training 

•  Format 

-  Live  or  webinar  (est.  2-4  hours) 

-  Combination  of  didactic  teaching  with  case  examples  and  vignettes 

•  Content/Learning  Objectives 

-  Communication  training 

•  Start  by  asking  about  any  concerns  (physical  and  BH)  regardless  of  self-report 

•  Evaluate  previous  treatment  where  problems  reported 
■  Strategies  to  elicit  self-disclosure 

-  Education  for  SMs 

•  Effective  statements  to  normalize  post-deployment  problems 

•  Information  on  effectiveness  and  availability  of  health  care  and  treatment  options 

-  Documentation  guidelines/compliance  rules 

•  When  to  use  open  ended  fields  (SM  just  documenting  problems  but  no  concerns; 
previous  treatment  adequate  thus  no  concern;  SM  reported  problems  not  on  self-report; 
etc.) 

•  Document  previous  treatment 

•  What  is  a  major  concern/minor  concern 

-  Decision  Rules  (i.e.,  rules  for  considering  a  positive  screen) 

•  Will  adapt  based  on  implementation  of  NDAA  Sec.  708  25 
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Component  2:  Decision  Support 

•  Decision  support  ~  est.  0-2  minutes  for  most  SMs,  up  to  5  minutes 
for  SMs  with  many  symptoms 

-  SM  specific  information  provided  during  PDHRA  interview 

-  Targeted  to  action:  determining  likely  need  for  referral 

-  Individualized  to  the  SM’s  self-report  responses 

•  Simple  ‘dashboard’  of  pertinent  information  easily  available  and 
already  interpreted  (example  on  next  slide) 

-  Relevant  PDHA  data  (e.g.,  combat  exposure,  deployments) 

-  Interpretive  summary  of  current  PDHRA  self-report 

•  Algorithms  by  subscale 

•  Enhanced  risk  based  on  multiple  subscales;  SM  factors 

•  Decreases  need  for  interpretation  of  data  so  provider  can  focus  on 
interaction 

-  Match  between  self-report,  risk  factors  and  SM  presentation? 

-  SM  just  documenting  concerns? 

-  Adequacy  of  previous  treatment  where  there  are  concerns? 
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Electronic  System  or  Paper 

•  Benefits  of  computer-based  system 

-  Automatic  recording  of  provider  viewing  of  decision-support  and  feedback  for  implementation  tracking 

-  Possibility  of  adding  simple  questions  for  research  purposes  (e.g.,  rating  of  SM  level  of  disclosure 
during  interview) 

•  Could  provide  algorithms  for  decision-support  on  paper 

-  Requires  provider  hand  completion 

-  Likely  decreased  fidelity  and  increased  burden 


Decision  Support  Algorithm:  Example 


Providers,  please  refer  to  the  PDHRA  to  complete  the  following  questions. 


1. 

2. 

3. 

4. 


Did  the  SM  endorse  any  TBI  symptoms  in  the  past  week  (Q 
9d)?  1 

Did  the  SM  endorse  “yes”  or  “unsure”  regarding  relationship 
conflicts  (Q  II)?2 

Were  at  least  2  PTSD  symptoms  (Q  1 2)  endorsed  on  the 
PDHRA  by  this  SM?  3 

Was  at  least  1  depression  symptom  (Q  14)  endorsed  “more 
than  half  the  days”  or  “nearly  every  day”  on  the  PDHRA  by  this 
SM?2 


Referral  Recommended 


1  Air  Force.  (2008).  Post-deployment  health  reassessment:  Application  user’s  guide.  Retrieved  October  5,  2008  from  Deployment  Health  Clinical  Center: 
http://www.pdhealth.mil/dcs/downloads/PDHRA_AUG.pdf 

2  Bliese,  P.,  Wright,  K.,  Adler,  A.,  Hoge,  C.,  &  Prayner,  R  (2005).  Post-deployment  psychological  screening:  Interpreting  and  scoring  DD  foim  2900.  Heidelberg, 

Germany:  US  Army  Medical  Research  Unit-Europe.  Research  Report  2005-003.  27 

3  Bliese,  P.,  Wright,  K.,  Adler,  A.,  Cabrera,  O.,  &  Hoge,  C.  (2008).  Validating  the  Primary  Care  Post  Traumatic  Stress  Disorder  Screen  and  the  post  traumatic  stress 
disorder  checklist  with  soldiers  returning  from  combat.  Journal  of  Consulting  and  Clinical  Psychology.  76:2,  272-81. 
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Decision  Support:  Mock  Up 

Service  Member:  George  Washington  Returned  from  Iraq  on  25  APR  2010 

ID:  002-22-1732  Deployment#:  4  Length:  12  mo. 

Combat  Exposure  from  PDHA-  In  Combat,  In  Danger 


1  or  more  problems 

Indicated  on 
PDHA 

Indicated  on 
PDHRA 

PTSD 

0 

Depression 

□ 

ET 

Traumatic  Brain  Injury 

S' 

0 

Relationship  Conflicts 

S' 

□ 

Alcohol  Use 

□ 

ET 

Potential  Problem 

PDHRA  Severity  Area 

CURRENT 

CURRENT 

CURRENT 

PREVIOUS 

CURRENT 


Example  1 


SM  Requested  Support 
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Decision  Support:  Mock  Up 

Service  Member:  Jordan  O’Neil  Returned  from  Afghanistan  on  28  MAR  2010 

ID:  197-44-5263  Deployment#:  2  Length:  8  mo. 

Combat  Exposure  from  PDHA-  In  Danger 


Indicated  on 


Area 

PDHA 

PTSD 

Depression 

Traumatic  Brain  Injury 

□ 

Relationship  Conflicts 

□ 

Alcohol  Use 

0 

Indicated  on 
PDHRA 

□ 

□ 

□ 

S' 

□ 


Potential  Problem 

PDHRA  Severity  Area 


PREVIOUS 


AUDIT-C:  2 


CURRENT 

PREVIOUS 


Example  2 


(N°) 


SM  Requested  Support 


YES 


29 
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Potential  Decision  Support  Elements 

PDHRA:  At  A  Glance 

Service  Member:  George  Washington 

ID:  002-22-1732 

Ultimate  Goal 

•Input:  problems  consistent  with 
diagnosable  condition 

Likelihood  thatSM  has  a 
PDHRA  area  problem  and  needs  a 

referral 

•Decision:  Refer  unless 

-  SM  simply  documenting 

PTSD  80% 

Depression  90% 

TBI  50% 

Relationship  Conflicts  10% 

Substance  Abuse  20% 

deployment-related  problems 

-  SM  currently  in  treatment  and 
states  no  need  for  further 
evaluation 

-  SM  denies  problem  warranting 
referral 

30 

Decision  Support:  Determining 
Data  Elements 

Risk  factors 

-  PDHA:  combat  exposure,  previous  self-reported  problems, 
location/number  of  deployments 


-  Presence  of  physical  and/or  exposure  concerns;  multiple  areas 
Support  rules  (developed  through  secondary  analysis  and  literature  review) 

-  Population-based  benchmarks  based  on  SM  self-report  (e.g.,  percentiles) 

•  Strengths:  Easily  available;  can  develop  profile-specific  based  on  SM  factors 
■  Limitations:  based  on  self-report  only;  may  change  over  time 

-  Concordance  of  SM  self-report  with  referral  alone,  referral  plus  diagnosis, 
or  diagnosis  alone 

•  Strengths:  Available  from  PDHRA  data  and  health  care  encounter  data 

•  Limitations:  cannot  evaluate  false  negatives;  questionable  gold  standards 

-  Algorithms  for  sensitivity/specificity  from  published  literature  (e.g.,  Bliese) 

•  Strengths:  Available  for  PC-PTSD;  established  gold  standard  of  semi-structured  interview 

•  Limitations:  Diagnosis-specific;  not  available  for  all  PDHRA  problem  areas 
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Component  3:  Weekly  Feedback 

•  Feedback  -  5  minutes  once/week  individually;  10  minutes 
once/week  as  a  group 

-  Available  each  week  for  review 

-  Each  provider  reviews  his  or  her  own  feedback;  site  manager 
reviews  all  providers 

-  Review  at  weekly  staff  meetings  to  enhance  peer  learning/support 

•  Aggregated  at  the  level  of  provider  and/or  installation 

•  Allows  for  continuous  quality  improvement 

-  Tracking  primary  PDHRA  outcomes 

-  Monitoring  adherence  to  training  guidelines 

-  Address  needs  for  additional  supports 

•  Content  linked  to  initial  training  and  decision  support 
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Feedback  Elements:  Mock  Up 
Referral  Completion  Quality  Control  Charts 


Increased  Referrals  Kept 


Decreased  Referrals  Kept 


PDHRA  Feedback 

All  Providers  Week  of  July  5,  2010  #  of  S Ms  screened:  600 


Rate  of  Referrals  Kept  Within  1  Week 

100%  -I 

90%  - 
80%  - 
70%  - 
60%  - 
50%  - 
40%  - 
30%  - 
20%  - 
10%  - 
0%  l 

Weekl  Week  2  Week  3  This  Week 

320  made  311  made  265  made  223  made 


PDHRA  Feedback 

All  Providers  Week  of  July  5,  2010  #ofSMs  screened:  600 


Rate  of  Referrals  Kept  Within  1  Week 

100% 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20% 

10% 

0% 

Weekl  Week  2  Week  3  This  Week 

318  made  312  made  360  made  350  made 


•  Could  provide  more  detailed  views  by  problem  type,  referral  type/time  frame,  etc. 

•  Information  to  lead  to  further  inquiry  and  communication  with  MTF 
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Feedback  Elements:  Mock  Up 
Provider-Specific  Detail 

PDHRA  Feedback 

Provider:  Fox,  Joe  Week  of:  July  5,  2010  #  of  SMs  screened:  600 

Rate  of  referrals  made  when  SM  had  positive  risk 
assessment 

50%  -  ^  ^  ^ 

40%  - 

30%  - 

20%  - 

10%  - 

Week  1  Week  2  Week  3  This  Week 

34 

Feedback:  Determining  Data 
Elements 

Input  Pleaset 

Organizational  Issues 

-  Ability  to  institute  weekly  staffing 

-  Ability  to  integrate  PDHRA  and  health  care  encounter  data 

Content  of  feedback 

-  Referral  completion  quality  control  charts  (to  help  ensure  providers  are 
not  simply  increasing  unwarranted  referrals) 

-  Comparison  to  decision  support  guidelines 

-  Documentation  compliance 

Unit  of  feedback 

-  Can  aggregate  at  any  level 

•  Individual  S Ms? 

-  Over  any  time  period 

■  Previous  week? 

Multiple  views  tailored  to  user  (provider,  site  manager) 

-  Main  dashboard:  Primary  information  ‘at  a  glance’ 

-  Detailed  views:  Digging  deeper 
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Program  Delivery  Issues, 

Options  for  producing  decision  support 

1 .  Housed  at  MedPros  :  Most  integrated 
VU  provides  algorithms  and  programming  support 
VU  provides  dates,  units,  locations  for  study  sample 
MedPros  makes  the  form  electronically  available  to  providers  along  with 
PDHRA 

De-identified  export  provided  to  VU  for  research 

2.  Housed  at  Installation:  Greater  site  demand 


3. 


•  VU  provides  program 

•  Installation  responsible  for  coordination  with  MedPros 

•  Installation  responsible  for  procedure  to  make  the  form  electronically 
available  to  providers  along  with  PDHRA 

•  De-identified  export  provided  to  VU  for  research 

Housed  at  VU:  Most  complex,  least  integrated 

•  De-identification  and  re-identification  process  required 

•  VU  coordinates  with  MedPros  and  installation  to  coordinate  data 
downloads  by  MedPros 

•  VU  manipulates  data  then  provides  forms  to  installation  (mail  or  secure 
download) 

•  Forms  are  paper  copies  contained  in  SM  SRP  packet 
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Program  Delivery  Issues  (cont) 

xr  Input  Please 

Delivery  of  weekly  feedback 

-  Another  weekly  process  for  MEDPROS  involvement 
post-interview 


-  Access  to  HCE  data  from  AHLTA  needed  to  include 
completed  referral  information 
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Determining  Sites 


In  gathering  preliminary  information  we  have 
established  relationships  with  Ft  Carson,  Ft  Benning, 
Ft  Campbell,  and  Ft  Riley.  POCs  have  indicated 
preliminary  interest  in  participating,  but  there  has  been 
no  contact  with  Installation  Commanders 
Considerations 

-  PDHRA  events  (scheduled  March  to  Sept  201 1 ) 

-  Provider  characteristics 

-  Site  agrees  to  pilot  as  QA  system 

Site  able  to  handle  potential  increased  referrals 

-  Site  ability  to  link  AHLTA  with  MEDPROS  (i.e.,  to  calculate 
rates  of  referral  completion) 

-  Site  agrees  to  minimal  changes  in  PDHRA  process 

•  Advance  scheduling  of  SMs  so  data  can  be  available 

•  PDHRA  self-report  completed  at  least  1  week  prior  to  interview 


38 


Next  Steps  for  Involvement  of 


Expert  Panel 


•  Suggested  modifications  to  program  design? 

•  Adapting  initial  training  based  on  NDAA  Sec.  708 

•  Determining  data  elements  for  decision  support  and 
weekly  feedback 

-  Data  analysis 

-  Literature  review 

•  Training  and  feedback  program  delivery 

-  Electronic  or  paper  form 

Options  for  producing  decision  support 

•  Site  selection 

•  Review  research  design 
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APPENDIX  E:  EXPERT  PANEL  MEMBERSHIP  ROSTER 
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Table  E.l  Expert  Panel  Members 


Name  Role 


COL  Charles  Engel 

Director,  DHCC  at  Walter  Reed  Army  Medical  Center,  Senior 
Scientist  at  the  Center  for  the  Study  of  Traumatic  Stress,  and 
Associate  Professor  and  Associate  Chair  at  the  Department 
of  Psychiatry  at  the  Uniformed  Services  University  School  of 
Medicine 

Dr.  Lucinda  Frost 

Senior  Mental  Health  Policy  Analyst,  Clinical  &  Program 

Policy,  Office  of  the  Asst  Secretary  of  Defense  (Health 

Affairs) 

CAPT  John  Golden 

Acting  Deputy  Director  Psychological  Health  Clinical 

Standards  of  Care,  DCoE 

Dr.  (Retired  COL)  Charles  Hoge 

Psychological  Health  Consultant  to  the  Army  Surgeon 

General 

CAPT  Sara  Kass 

Bureau  of  Medicine  (Navy)  and  Navy  Family  Practice 

Dr.  (Retired  COL)  John  Kugler 

Deputy  Chief  Medical  Officer,  TRICARE  Management  Activity 

Lt  Col  Hans  Ritschard 

Director,  DoD  Psychological  Health  Strategic  Operations, 

Force  Health  Protection  and  Readiness 

COL  Louis  Smith 

Senior  Physician's  Assistant,  DeLorenzo  Army  Health  Clinic, 
Pentagon 

Dr.  Brian  Sugden 

Project  Manager,  Reserve  Health  Readiness  Program 

Force  Health  Protection  and  Readiness 
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APPENDIX  F:  EXTERNAL  MEETING  SCHEDULE  FOR  YEAR  ONE 
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Table  F.l  includes  Planning  Meetings  (Task  6),  Expert  Panel  Meetings  (Task  5),  and  other  meetings 
conducted  for  educational  or  informational  purposes  during  Year  1. 


Table  F.  1  External  meeting  schedule  for  year  one 


Date 

Format 

Purpose 

Attendees 

01-OCT-09 

Teleconference 

Planning— Discuss  how  VU  might  keep 
previously  received  data 

VU,  FHP&R 

02-OCT-09  to 

29-DEC-09 

No  weekly  planning  meetings  with  FHP&R  were  held  during  this  time,  because  VU  had 
not  been  assigned  a  project  manager. 

26-OCT-09 

Teleconference 

Informational— Discuss  possibility  of 
retaining  data 

VU,  AFHSC 

23-DEC-09 

Teleconference 

Informational— Discuss  MEDPROS  data 
request  procedures 

VU,  MEDPROS  (Audrey 
Luken) 

30-DEC-09 

Teleconference 

Planning— Discuss  AFHSC  data 

VU,  FHP&R 

05-JAN-10 

Teleconference 

Informational/Planning— Discuss  roles  of 
MRMC  and  FHP&R  for  this  award. 

VU,  MRMC 

12-JAN-10 

Teleconference 

Informational— Discuss  data  sharing 
documentation  requirements 

VU,  MRMC 

12-JAN-10 

Teleconference 

Informational— Identify  Navy  contact  for 

VU  data  request 

VU,  Navy 

12-JAN-10 

Teleconference 

Informational— Discuss  data  available  in 

Army  database 

VU,  Army 

14-JAN-10 

In-Person 

Planning/Presentation— Present  and 
discuss  findings  from  VU's  previous  contract 
that  are  relevant  to  current  grant. 

VU,  FHP&R, 
representatives  from 
multiple  agencies  and 
services  (Mr.  Ron 
Crusher,  Col  Joyce 
Adkins,  LTC  Jim 

Bennett,  Dr.  Terry 
Washam,  Dr.  Fred 
Glogower,  Ms.  Janet 
Bouncy,  Mr.  Joe 

Pedone,  Mr.  Larry 
Verbiest,  Mr.  Darnell 
Neal,  Mr.  Billy  Dean, 

LTC  Ball,  Mr.  Erik 

Noell,  Ms.  Betty  Tally, 
LTC  Tracy  Neal- 
Walden,  Mr.  Rich 
Roeske,  Mr.  Jim  Reis, 
General  Williams.) 

14-JAN-10 

In-Person 

Planning— Discuss  progress/next  steps 

VU,  FHP&R 
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Date 

Format 

Purpose 

Attendees 

14-JAN-10 

In-Person 

Informational/Approvals— Sign  DUA  and 
discuss  data  sharing. 

VU,  FHP&R,  AFHSC 

19-JAN-10 

Teleconference 

Planning— Discuss  Purdue  contract 

VU,  Purdue 

24-FEB-10 

Teleconference 

Presentation— Present  findings  from  VU's 
previous  contract  to  FHP&R  weekly  journal 
club. 

VU,  FHP&R 

03-MAR-10 

Teleconference 

Presentation— Present  overview  of  current 
project 

VU,  FHP&R,  DCoE 

05-MAR-10 

Teleconference 

Planning— Discuss  possibility  of  VU 
contributing  to  NDAA  activities 

VU,  FHP&R 

08-MAR-10 

Teleconference 

Informational— Discuss 
possibility/procedures  for  modifying  SOW 
as  relevant  to  NDAA 

VU,  MRMC 

10-MAR-10 

Teleconference 

Planning— Discuss  possibility  of  VU 
contribution  to  NDAA  provider  training 

VU,  FHP&R 

23-MAR-10 

Teleconference 

Informational— Discuss  actuarial  modeling 

VU,  LTC  Paul  Bliese 

29-MAR-10 

Teleconference 

Planning— Discuss  NDAA 

VU,  FHP&R 

06-APR-10 

Teleconference 

Planning— Develop  project  coordination 

VU,  Purdue 

09-APR-10 

Teleconference 

Planning— Update  and  check-in 

VU,  FHP&R 

20-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA-specific  provider 
training. 

VU,  Camp  Pendleton 

23-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA-specific  provider 
training. 

VU,  Ft  Carson 

23-APR-10 

Teleconference 

Planning— Clarify  relative  roles  of  VU, 

FHP&R,  and  MRMC  for  project 

VU,  MRMC,  FHP&R 

26-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA-specific  provider 
training. 

VU,  Fort  Benning 

27-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA 

VU,  Fort  Campbell 

27-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA 

VU,  Fort  Riley 

28-APR-10 

Teleconference 

Informational— Inquire  about  practices  and 
needs  regarding  PDHRA 

VU,  Fort  Carson 

29-APR-10 

Teleconference 

Weekly  Planning— This  was  the  first 
regularly  scheduled  weekly  planning 
meeting.  Future  schedule  discussed. 

VU,  MRMC,  FHP&R 

04-MAY-10 

Teleconference 

Weekly  Planning— Discuss  conversations 
with  installations  and  Expert  Panel  selection 

VU,  FHP&R 
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Date 

Format 

Purpose 

Attendees 

12-MAY-10 

Teleconference 

Weekly  Planning -Discuss  Expert  Panel 
Selection  and  AFHSC  DUA 

VU,  FHP&R 

26-MAY-10 

Teleconference 

Weekly  Planning— Discuss  FHP&R's 
progress  contacting  potential  Expert  Panel 
members  and  plans  for  intervention  design. 

VU,  FHP&R,  MRMC, 
Purdue 

02-JUN-10 

Teleconference 

Weekly  Planning— Discuss  project  and 

NDAA  timelines  and  Expert  Panel  selection. 

VU,  FHP&R,  MRMC, 
Purdue 

09-JUN-10 

Teleconference 

Weekly  Planning  -Discuss  procedures  for 
contacting  installations 

VU,  MRMC 

14-JUN-10 

Teleconference 

Weekly  Planning  -Discuss  DUA  and  FHP&R 
progress  contacting  potential  Expert  Panel 
Members 

VU,  FHP&R,  MRMC 

18-JUN-10 

Teleconference 

Informational— Discuss  feasibility  issues 
regarding  data  manipulation 

VU,  MEDPROS 

23-JUN-10 

Teleconference 

Weekly  Planning  -Discuss  project  timeline 
and  DUA 

VU,  FHP&R 

25-JUN-10 

Teleconference 

Approvals— Discuss  revised  AFHSC  DUA 

VU,  VU's  Division  of 
Sponsored  Research 
(DSR) 

30-JUN-10 

Teleconference 

Weekly  Planning— Plan  for  first  Expert 

Panel  meeting  and  discuss  PDHRA 
schedules  for  Mar-Sept  2011. 

VU,  FHP&R,  MRMC, 
Purdue 

07-JUL-10 

Teleconference 

Approvals— Discuss  DSR's  concerns  with 
wording  in  the  data  request 

VU,  VU  DSR,  FHP&R 

08-JUL-10 

Teleconference 

Weekly  Planning— VU  presented  proposed 
slides  for  July  21  Expert  Panel  presentation 
for  feedback  from  FHP&R 

VU,  FHP&R,  Purdue 

12-JUL-10 

Teleconference 

Planning— Discuss  ways  of  integrating 
Purdue's  data  into  a  decision  support 
system  for  VU's  intervention. 

VU,  Purdue 

14-JUL-10 

Teleconference 

Weekly  Planning— Discuss  possibility  of 
obtaining  data  from  MedPros. 

VU,  FHP&R,  Purdue 

19-JUL-10 

Teleconference 

Weekly  Planning— VU  presented  research 
design  slides  for  FHP&R  review  and 
approval. 

VU,  FHP&R,  Purdue 

21-JUL-10 

In-Person 

Expert  Panel— VU  introduced  proposed 
PDHRA  provider  training  and  feedback 
program  so  panel  members  could  offer 
ideas  and  insights  on  development. 

VU,  FHP&R,  Expert 

Panel 

21-JUL-10 

In-Person 

Planning— Review  Expert  Panel  feedback, 
discuss  next  steps,  review  NDAA  training 
development. 

VU,  FHP&R 
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Date 

Format 

Purpose 

Attendees 

12-AUG-10 

Teleconference 

Planning— Develop  Project  Coordination 

VU,  Purdue 

18-AUG-10 

Teleconference 

Weekly  Planning -Discuss  DUA,  NDAA 
updates,  and  new  Expert  Panel  Member 

COL  Smith)  recruited  by  FHP&R.  Discuss 
scheduling  next  Expert  Panel  Meeting. 

VU,  FHP&R 

20-AUG-10 

Teleconference 

Approvals— Clarify  DSR's  concerns  with 
AFHSC  DUA. 

VU,  VU  DSR 

25-AUG-10 

Teleconference 

Approvals— Discuss  DSR's  concerns  with 
data  request  wording,  and  ways  to  address 

concerns. 

VU,  VU  DSR,  FHP&R 

25-AUG-To 

Teleconference 

Weekly  Planning  -Discuss  AFHSC  DUA 

VU,  FHP&R,  Purdue 

26-AUG-10 

Teleconference 

Informational— to  view  a  demo  of  DHCC's 
RESPECT-Mil  system  and  discuss  ways  this 
might  inform  VU's  work. 

VU,  FHP&R,  DHCC 
(Expert  Panel  Member 
COL  John  Engel) 

01-SEP-10 

Teleconference 

Weekly  Planning— Discuss  possibilities  and 
procedures  for  obtaining  data  through 
MEDPROS. 

VU,  FHP&R, 

MEDPROS,  MRMC, 
Purdue 

01-SEP-10 

Teleconference 

Planning— Develop  project  coordination 

VU,  Purdue 

17-SEP-10 

Teleconference 

Weekly  Planning— share  ideas  on  most 
effective  way  to  conduct  intervention  based 
on  information  received  about  NDAA  policy. 

VU,  FHP&R,  Purdue 

17-SEP-10 

Teleconference 

Informational— Discuss  ways  in  which 
RESPECT-Mil  might  inform  VU's  work. 

VU,  DHCC  (Expert 

Panel  Member  COL 

John  Engel,  Ms.  Sheila 
Barry) 

29-SEP-10 

Teleconference 

Weekly  Planning— Discuss  next  steps  and 
goals  of  in-person  intervention  design 
meeting  with  Purdue  the  next  day  (Year  2) 

VU,  FHP&R 

104 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


APPENDIX  G:  PURDUE  LITERATURE  REVIEW  -  PTSD  PREVALENCE  AND 

RISK/PROTECTIVE  FACTORS 
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One  of  the  goals  of  our  secondary  analysis  is  to  develop  a  comprehensive  estimate  of  PTSD 
prevalence  among  OIF/OEF  veterans,  using  information  from  the  PDFIA,  PDHRA,  and  medical 
records  data.  As  such,  we  review  here  various  current  estimates  using  different  samples  and 
methodologies  to  establish  the  range  of  current  estimates  and  to  explore  the  impact  of 
different  methodologies  and  definitions  on  those  estimates.  To  date,  no  definitive  count  is 
available  of  service  members  and  veterans  who  were  ever  deployed  to  Iraq  or  Afghanistan  and 
are  impaired  with  PTSD,  either  diagnosed  or  undiagnosed.  In  a  special  guest  editorial,  Bass  and 
colleagues  discuss  three  main  problems  with  the  current  prevalence  rates  of  TBI  and  PTSD  in 
the  OIF/OEF  population.  First,  the  studies  generally  report  the  percentage  of  service  members 
who  screen  positive  for  TBI  or  PTSD,  not  those  who  have  been  diagnosed  with  the  condition  by 
an  appropriately  trained  medical  provider.  Second,  the  study  samples  are  not  representative  of 
the  entire  ever-deployed  military  population.  Most  of  published  studies  on  TBI  and  PTSD  are 
often  combat  troops  in  the  Army  or  Marine  Corps,  less  frequently  including  support  personnel 
or  personnel  from  the  Navy  or  Air  Force.  Third,  the  degree  of  impairment  for  service  members 
who  have  or  have  had  TBI  or  PTSD  is  unknown  (Editorial,  2009) 

Prevalent  estimates  vary  with  the  diagnostic  criteria  applied  (cut-offs),  the  sample  studied,  and 
the  assessment  procedures  used.  For  example: 

1)  Millken  et  al.  (2007),  reported  that  using  a  threshold  of  3  out  of  4  rather  than  2  out  of  4 
almost  halved  their  estimate  to  6.12  from  11.8  percent  for  active  Army  soldiers  immediately 
post  deployment.  Recent  reports  indicated  that  service  members  were  more  than  2  times  as 
likely  to  report  mental  health  concerns  3  to  4  months  after  returning  from  deployment 
compared  with  reporting  immediately  on  return  (Bliese,  et  al.,  2004).  This  has  led  to  a  decision 
to  expand  the  scope  of  the  current  US  military  screening  program  to  include  a  repeat  measure 
at  90  to  180  days  after  deployment  (Deployment  Clinical  Health  Center  (DFICC),  2010). 

2)  As  to  screening  tools,  a  number  of  different  instruments  have  been  developed  over  the  last 
two  decades  to  measure  or  diagnose  PTSD  in  adults  including  Mississippi  Scale  for  Combat- 
related  PTSD  (M-PTSD),  PTSD  Checklist  (PCL),  and  alternative  versions  of  the  PTSD  module  of 
the  Diagnostic  Interview  Schedule  (DIS),  among  others.  Screening  tools  may  have  limited 
predictive  value  for  individuals. 

3)  The  assessment  timeframe  also  contributes  to  varying  estimate  rates  of  PTSD.  Screening 
immediately  upon  return  has  low  specificity  and  positive  predictive  value.  Positive  predictive 
value  is  highly  dependent  on  the  prevalence  of  the  disorders,  and  a  positive  predictive  value 
would  be  expected  to  be  lower  for  screening  tests  applied  on  return  from  deployment 
compared  with  3  to  4  months  later.  In  three  wave  longitudinal  study,  soldiers  were  found  to  be 
at  an  increased  risk  for  developing  PTSD  over  time  (Wolfe,  Erickson,  Sharkansky,  King,  &  King, 
1999). 

Studies  of  veterans  conducted  years  after  their  service  ended  have  shown  a  prevalence  of 
current  PTSD  of  15  %  among  Vietnam  veterans  and  2-10%  among  veterans  of  the  first  Gulf  War 
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(Kang,  Natelson,  Mahan,  Lee,  &  Murphy,  2003;  The  Iowa  Persian  Gulf  Study,  1997).  The 
National  Vietnam  Veterans  Readjustment  Study  shows  19%  Vietnam  veterans  reported  lifetime 
PTSD  (Dohrenwend,  et  al.,  2006;  Hoge,  et  al.,  2004).  Similarly,  Hoge  (2004)  found  as  many  as 
17-19%  of  active  duty  component  soldiers  screened  positive  for  PTSD,  depression,  or  anxiety 
upon  returning  from  OEF/OIF. 

Because  estimates  of  current  and  lifetime  PTSD  are  rather  divergent  and  differ  by  sample, 
timeframe,  and  screening  tool,  we  will  now  review  estimation  procedures,  focusing  on  PTSD 
among  veterans  from  Iraq  and  Afghanistan  only. 

Data 

Hoge  et  al.  (2004):  Members  of  four  U.S.  combat  infantry  units  (three  Army  units  and  one 
Marine  Corp  unit)  either  before  their  deployment  to  Iraq  or  three  to  four  months  after  their 
return  from  combat  duty  in  Iraq  or  Afghanistan.  To  assess  the  representativeness,  they 
compared  the  demographic  characteristics  of  respondents  with  those  all  active-duty  Army  and 
Marine  personnel  deployed  to  OIF  and  OEF  using  the  Defense  Medical  Surveillance  System. 
However  it  still  cannot  be  generalized  to  other  components,  such  as  NGR  (National  Guard 
Reserve). 

Hoge  et  al.  (2006):  The  US  military  has  conducted  population-level  screening  for  mental  health 
problems  among  all  service  members  returning  from  deployment  to  OEF,  OIF  and  other 
locations,  that  is,  PDHA(Post-Deployment  Health  Assessment)  during  the  first  year  after 
return.  This  study  obtained  data  from  PDHA. 

Seal  et  al.  (2007),  Seal  et  al.  (2009):  Using  national  VA  (Veteran  Affairs)  data.  Study  population 
consisted  only  of  OIF  and  OEF  veterans  who  were  first  time  uses  of  VA  services  after  their  OEF 
or  OIF  military  service.  Estimated  rates  will  be  bias  in  that  rates  of  preexisting  or  new  mental 
diagnoses  may  be  higher  among  OIF  and  OEF  veterans  who  use  the  VA  health  care  system 
compared  with  non  users.  Results  are  not  generalizable  to  all  veterans  of  OEF/OIF  service 
because  no  data  on  veterans  who  have  not  accessed  VA  care. 

Milliken  et  al.  (2007):  The  DMSS  (Defense  Medical  Surveillance  System);  Both  PDHA  (Post 
Deployment  Health  Assessment)  and  PDHRA  (Post  Deployment  Health  Assessment)  is  a  part  of 
the  soldier's  permanent  medical  recode,  and  an  electronic  copy  is  integrated  into  the  DMSS. 

For  active  component  solders,  all  health  care  that  they  receive  in  military  treatment  facilities  is 
reported  to  DMSS  including  clinic  type  and  diagnoses. 

Rand  study  "Invisible  Wounds  of  War"  (2008):  It  is  one  of  the  most  widely  cited  studies  of 
OIF/OEF  veterans'  health.  This  study  is  based  on  interviews  of  a  representative  sample  of  1965 
returning  troops  from  Iraq  and  Afghanistan  from  August  2007  to  Jan  2008.  The  survey  of 
recently  returned  service  members  drew  from  the  population  of  all  of  those  who  have  been 
deployed  for  OIF  and  OEF,  regardless  of  Service  branch,  component,  or  unit  type.  The  survey 
used  random  digit  dialing  to  reach  a  representative  sample  within  the  targeted  locations. 


107 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


[Gulf  study  -  not  OIF/OEF:  (Kang,  et  al.,  2003)  Data:  the  Defense  of  Manpower  Data  Center 
provided  military  and  demographic  information.  A  stratified  sampling  method  was  adopted  to 
ensure  that  each  subgroup  was  adequately  represented  among  the  15,000  Gulf  Veterans.  The 
entire  population  of  troops  deployed  to  the  Gulf  area  was  stratified  by  gender,  unit  component 
(active,  reserve,  and  National  Guard),  and  branch  of  service  (Army,  Navy,  Airforce,  and  Marine 
Corps).  Random  stratified  sampling.  The  results  shows  that  the  veteran  positive  for  PTSD 
criteria  more  likely  to  be  female,  older,  non-white,  in  the  enlisted  ranks,  and  in  the  Army  and 
National  Guard.  10.1  percent,  the  population  prevalence  a  PTSD.  ] 

Screening  Tool  for  PTSD 

PTSD  symptoms  were  frequently  assessed  using  PTSD  Checklist  Military  version  (PCL-M) ,  a  17 
self-reported  items  rated  from  1  to  5  pts  on  a  Likert-type  scale.  The  PTSD  cut-off  requires  a 
global  score  of  50  and  the  symptoms  endorsed  at  a  moderate  or  high  level.  PTSD  diagnosis  was 
evaluated  with  the  use  of  the  17-item  National  Center  for  PTSD  checklist  of  the  Department  of 
Veterans  Affairs  (Hoge,  et  al.,  2004;  Hoge  et  al .,  2004 ;  Kang ,  et  al .,  2003;  The  Iowa  Persian  Gulf 
Study ,  1997) 

Hoge  et  al  (2004):  The  PDHA  includes  a  4-item  screen  for  PTSD,  developed  by  National  Center 
for  PTSD  for  primary  care  settings  (PC-PTSD).  They  use  2  or  3  items  out  of  the  4-items  PTSD 
screen  as  a  cutoff. 

Seal  et  al.  (Seal,  et  al.,  2007;  Seal,  et  al.,  2009)  utilize  ICD-9-CM  codes  (International 
Classification  of  disease.  Ninth  Revision  Clinical  Modification). 

Milliken  et  al.  (2007):  the  four-question  Primary  Care-PTSD  screen  (PC-PTSD)  asks  respondents 
whether  they  experienced  certain  symptoms  in  the  past  month  due  to  a  traumatic  event.  A 
positive  screen  was  recorded  if  the  respondent  endorses  at  least  two  of  the  four  PTSD  related 
items. 

Rand  study,  "Invisible  Wounds  of  War"  (2008):  based  on  the  responses  to  a  telephone 
administration  of  the  PTSD  Checklist  (PCL).  The  PCL  module  contains  17  questions  covering  the 
three  symptom  clusters  of  PTSD:  reexperiencing  trauma,  avoidance,  and  hyper-arousal.  Each 
question  is  scored  on  a  5-point  scale  with  a  maximum  possible  score  of  85.  One  scoring  method 
assesses  a  probable  case  of  PTSD  if  the  respondent  endorses  with  a  value  of  at  least  3  on  the  5- 
point  scale.  The  Rand  study  asserted  that  the  PCL  scored  via  this  method  (cluster  method)  has  a 
sensitivity  of  1.00  and  a  specificity  of  0.92. 

Time  Frame 

Hoge  et  al  (2006) ,  Seal  et  al  (Seal,  et  al.,  2007;  Seal,  et  al.,  2009)  and  Miliken  et  al  (2007)  utilize 
existing  data  that  contain  mental  health  information  (including  PTSD)  and  estimate  prevalence 
of  PTSD;  Hoge  et  al.  (2004)  conducted  a  survey  to  assess  mental  health  among  veterans  from 
OIF  and  OEF.  The  research  shows  that  majority  of  people  in  whom  PTSD  develops  meet  the 
criteria  for  the  diagnosis  of  this  disorder  within  the  first  three  months  after  the  traumatic  event. 
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Hoge  et  al.  (2004)  administered  the  surveys  three  to  four  months  after  the  subjects  had 
returned  from  deployment  and  at  least  six  months  after  the  heaviest  combat  operations. 

Seal  et  al.  (Seal,  et  al.,  2007;  Seal,  et  al.,  2009)  determined  the  cumulative  prevalence  of  mental 
health  diagnoses  in  the  VA  system  during  the  entire  study  period.  The  study  period  consisted  of 
24  calendar  quarters  from  April  2002  through  March  2008. 

Millken  et  al.  (2007):  The  military's  health  assessments  were  given  to  service  members 
immediately  upon  their  return  home  and  again  about  6month  later  (PDHA/PDHRA).  Note  that 
this  study  is  based  on  existing  medical  surveillance  data. 

[In  the  study  of  Gulf  War  veterans  (Kang,  et  al.,  2003),  sampling  was  obtained  from  database 
and  mental  health  related  survey  was  conducted.  Data  was  obtained  from  the  Defense  of 
Manpower  Data  Center  and  provided  military  and  demographic  information.  A  stratified 
sampling  method  was  adopted  to  ensure  that  each  subgroup  was  adequately  represented 
among  the  15,000  Gulf  Veterans.  The  entire  population  of  troops  deployed  to  the  Gulf  area  was 
stratified  by  gender,  unit  component  (active,  reserve,  and  National  Guard),  and  branch  of 
service  (Army,  Navy,  Air  force,  and  Marine  Corps).  Random  stratified  sampling.  The  results 
showed  that  the  veterans  positive  for  PTSD  criteria  were  more  likely  to  be  female,  older,  non¬ 
white,  in  the  enlisted  ranks,  and  in  the  Army  and  National  Guard.  10.1  percent,  the  population 
prevalence  of  PTSD.] 

Results  (Estimated  Rates) 

Hoge  et  al.  (2004)  Based  on  strict  definition,  rates  ranged  from  6.2%  to  12.9%;  based  on  broad 
definition,  they  ranged  from  11.5%  to  19.9%. 

Hoge  et  al.  (2006)  reported  that  of  303,905  army  and  marine  veterans,  35%  accessed  military 
mental  health  services  within  1  year  of  returning  home.  It  is  unclear  what  proportion  of  the 
35%  involved  PTSD.  Using  either  2  or  3  items  out  of  the  4  item  PTSD  screen  as  a  cut-off, 
between  4.7%  and  9.8%  screened  positive  in  this  population.  Hoge  et  al.  (2006)  stated 
regarding  the  differences  in  rates  between  two  studies:  "the  difference  in  scales  used,  whether 
the  surveys  included  identifiers,  and  particularly  differences  in  the  timing  of  the  surveys." 

Seal  et  al.  (2007)  The  frequency  of  ICD-9-CM  PTSD  Diagnoses  observed  among  OEF/OIF 
veterans  in  this  study  is  13%. 

Seal  et  al.  (2009)  2-year  period  prevalence  of  new  PTSD  diagnoses  is  18.2%  among  final  cohort 
of  OIF  and  OEF  veterans  entering  the  VA  health  care  system  Jan  2006  and  followed  2  years. 

Milliken  et  al.  (2007)  examined  military  PDHA  and  estimated  that  12  percent  of  active  Army 
personnel  and  13  percent  of  Army  reservists  screened  positive  for  PTSD  symptoms  immediately 
upon  returning  from  deployment.  Those  rates  rose  to  17  and  25  percent  of  active  and  reserve 
soldiers  respectively  at  the  PDHRA. 
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(Rand,  2008).  18.5%  of  all  returning  SMs  meet  criteria  for  either  PTSD  or  depression;  14%  of 
returning  SMs  currently  meet  criteria  for  PTSD.  If  these  numbers  are  representative,  then  of  the 
1.64  million  deployed  to  date,  the  study  estimates  that  approximately  300,000  veterans  who 
have  returned  from  Iraq  and  Afghanistan  are  currently  suffering  from  PTSD  or  major 
depression.  JRRD  Guest  Editorial  stated  that  "RAND's  survey  was  not  representative  of  the 
ever-deployed  military  population.  It  did  not  account  for  features  of  and  events  occurring 
during  deployment.  The  weighted  percentage  of  service  members  who  claimed  to  have 
returned  from  their  second  or  higher  deployment  was  47  percent;  population  based  data  from 
DOD  (Department  OF  Defense)  indicated  that  through  Feb.  2008  only  36  percent  of  service 
members  who  had  deployed  to  either  Iraq  or  Afghanistan  did  so  multiple  times." 

Risk  Factors 

Overall,  not  many  of  these  studies  reviewed  above  explicitly  addressed  risk  and  protective 
factors  of  PTSD.  Those  factors  are  included  as  control  variables  in  most  of  studies.  Flence,  in 
addition  to  the  studies  reviewed  above  that  specifically  estimate  prevalence  or  incidence  rates 
of  PTSD,  we  also  review  studies  below  that  focus  on  risk  and  protective  factors,  because,  in 
addition  to  estimating  prevalence  of  PTSD,  we  are  also  interested  in  examining  service-level, 
family-level,  and  individual-level  risk  and  protective  factors. 

OVERVIEW 

Meta-analyses  have  identified  several  risk  factors  for  PTSD  in  both  the  military  and  general 
population  (Brewin,  Andrews,  &  Valentine,  2000).  Pre-trauma  factors  such  as  family  psychiatric 
history,  personal  psychiatric  history,  and  childhood  abuse  most  consistently  predicted  PTSD, 
regardless  of  the  population  studied  or  the  methods  used.  Other  important  risk  factors  for 
veterans  included  younger  age  at  the  time  of  the  trauma,  less  education,  minority 
race/ethnicity,  and  family  adversity.  Trauma  severity  and  post-trauma  social  support  were 
more  important  in  military  than  in  civilian  samples. 

In  studies  of  PTSD  specifically  in  veterans,  predisposing  factors  have  included  minority 
race/ethnicity,  lower  education,  younger  age  at  exposure,  lower  SES,  family  problems  in 
childhood,  pre-trauma  psychopathology,  and  childhood  behavior  problems.  Event 
characteristics  that  increase  risk  for  PTSD  include  type  of  trauma,  amount  of  exposure,  injury, 
involvement  in  atrocities,  and  perceived  life  threat.  Post  event  factors  that  predict  PTSD  in 
veterans  are  low  social  support,  negative  homecoming  experiences,  and  poor  coping  and 
posttraumatic  life  events.  High  social  support  may  be  a  protective  factor  against  developing 
PTSD  in  the  general  population  (Brewin,  et  al.,  2000;  Ozer,  Best,  Lipsey,  &  Weiss,  2003)  and 
military  settings  (Green,  Grace,  Lindy,  Gleser,  &  Leonard,  1990;  Green  &  Berlin,  1987;  King, 

King,  Fairbank,  Keane,  &  Adams,  1998;  Solomon,  Margalit,  Waysman,  &  Bleich,  1991 ). 

Service-level  factors 

PTSD  in  SMs  has  been  found  to  be  associated  with  being  in  the  enlisted  ranks,  in  the  Army 
(versus  Marine  Corp,  Air  Force,  Navy)  and  National  Guard  (versus  Active  duty.  Reserve,  Kang  et 
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al  2003).  Studies  of  veterans  from  the  Gulf  War  similarly  show  that  non-active  duty  personnel 
(i.e.,  Reserve  and  National  Guard)  reported  experiencing  more  PTSD  symptoms  than  active  duty 
personnel  (Stretch,  Marlow,  &  Wright,  1996)  (Stretch  et  al.,  1996).  In  terms  of  military  rank, 
enlisted  personnel  have  reported  more  PTSD  symptoms  than  officers  (Adler,  Vaitkus,  &  Martin, 
1996).  A  more  recent  study  shows  that  SMs  enlisted  (versus  officer)  and  serving  in  the  US 
Marine  Corps  or  US  Army  (versus  Air  Force,  Navy,  or  Coast  Guard)  are  more  likely  to  have  PTSD, 
which  differs  from  other  studies  that  have  found  lower  rates  of  PTSD  in  the  Marine  Corps 
(Wells,  et  al.,  2009).  PTSD  is  also  more  common  in  short-term,  lower  rank  Army  personnel 
(Iverson,  et  al.,  2008;  Riddle,  et  al.,  2007).  Iversen  et  al's  (2008)  study  on  UK  forces  found  the 
association  between  PTSD  and  component,  with  the  Royal  Navy  and  Army  experiencing  greater 
symptoms  than  those  in  the  Royal  Air  Force.  The  Royal  Marine  Commandoes  (Elite  Naval 
Service  combat  troops)  had  a  relatively  low  prevalence  of  PTSD. 

Seal  et  al  (2009)  examined  the  association  of  service  component  with  the  development  of  PTSD 
more  specifically  by  taking  age  distribution  in  each  component  into  account.  Seal  et  al.  (2009) 
found  that  in  multivariate  analyses  adjusted  for  socio-demographic  and  military  service 
characteristics  stratified  by  component  type,  the  youngest  active  duty  veterans,  aged  16-24 
years,  were  at  the  highest  risk  for  diagnoses  of  PTSD  compared  with  active  duty  veterans  older 
than  40  years.  Younger  active  duty  veterans  likely  had  greater  combat  exposure  as  a  function  of 
lower  rank,  which  may  explain  higher  rates  of  mental  health  diagnoses.  Among  active  duty 
veterans,  proxies  for  combat  exposure-rank,  branch,  and  multiple  deployments  were 
independently  associated  with  higher  risk  of  PTSD.  By  contrast,  older  National  Guard  and 
reserve  veterans  older  than  30  years  were  at  higher  risk  for  PTSD  and  depression  than  were 
younger  National  Guard  and  reserve  veterans.  One  explanation  for  greater  distress  among 
older  national  guard  and  reserve  members  is  that,  when  called  to  arms,  they  are  more  likely 
established  in  civilian  occupations;  have  family,  social,  and  community  ties;  and  may  have  had 
less  preparation  for  combat,  making  their  transition  to  the  war  zone  and  then  home  again  more 
stressful  (Friedman,  2005;  Jacobson,  et  al.,  2008). 

The  finding  on  higher  rate  of  PTSD  for  NGR  is  also  shown  in  studies  on  the  (course  of)  PTSD 
development.  NGR  (National  Guard/Reserve)  troops  are  at  increased  risk  for  the  development 
of  emotional  and  psychological  complications  compared  to  active  duty  troops  (M HAT-II,  2005). 
Milliken  and  colleagues  (2007)  found  that  rates  of  PTSD  and  depression  more  than  doubled 
among  NGR  component  soldiers  between  initial  Post-Deployment  Health  Assessment  and  the 
Post-Deployment  Health  Reassessment  conducted  about  6  months  later.  Another  recent  study 
also  shows  that  the  increase  in  emotional  problems  over  time  for  NGR  soldiers  exceeded  the 
rates  found  in  regular  active  duty  service  members  (Polusny,  et  al.,  2009).  This  risk  for  the 
development  of  psychiatric  disorders  appears  to  increase  at  a  greater  rate  for  NGR  soldiers  in 
the  months  and  years  following  deployment.  The  increase  in  emotional  problems  over  time  for 
NGR  soldiers  exceeded  the  rates  found  in  regular  active  duty  component  service  members. 
Studies  for  the  veterans  from  the  Gulf  War  have  shown  similar  results.  For  example,  in  a  three- 
wave  longitudinal  study  of  2,949  Gulf  War  I  veterans,  Wolfe  and  colleagues  found  that  NGR 
soldiers  were  at  increased  risk  for  developing  PTSD  over  time  (Wolfe,  et  al.,  1999).  Initially  at 
time  1,  when  soldiers  were  assessed  about4-5  days  following  their  return  from  deployment  to 
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Gulf  War  I,  NGR  status  was  not  associated  with  PTSD  symptoms.  However,  NGR  status 
independently  contributed  to  the  development  of  PTSD  2  years  later  in  this  same  cohort. 
Following  deployment,  NGR  component  soldiers  may  face  unique  reintegration  challenges  as 
they  transition  back  to  civilian  roles.  Compared  to  active  duty  soldiers,  NGR  soldiers  tend  to  be 
older  and  may  be  more  likely  to  have  left  family  and  civilian  work  responsibilities  outside  the 
military. 

As  a  result,  NGR  troops  may  face  significantly  greater  familial  and  occupational  strain  both 
during  and  following  deployment,  and  these  challenges  may  contribute  to  NGR  soldier's 
elevated  risk  for  mental  health  difficulties  post  deployment.  For  instance,  post  deployment 
stressful  life  events  have  been  shown  to  be  associated  with  higher  rates  of  PTSD  (Green,  et  al., 
1990).  Further,  because  they  have  not  embedded  with  their  military  units  following  a  combat 
deployment,  NGR  personnel  may  also  have  lower  levels  of  support  from  social  and  occupational 
peers,  which  may  also  increase  risk  for  PTSD  (King,  et  al.,  1998) 

Finally,  Keane  (2006)  found  the  number  of  firefights,  frequency  and  intensity  of  combat,  injury, 
rank,  component,  military  branch,  combat  exposure,  intensity  and  duration  of  combat,  the 
extent  of  physical  injury,  and  adjustment  during  military  service  to  be  associated  with  PTSD. 

Family-community-level  factors 

The  association  between  social  support  and  the  development  PTSD  is  very  robust  in  combat 
veterans  compared  to  civilians  exposed  to  interpersonal  violence  (Brewin,  et  al.,  2000).  Vietnam 
veterans  who  report  active  engagement  in  the  community  are  less  likely  to  have  PTSD  (Koenen, 
Stellman,  Stellman,  &  Sommer,  2003).  Sutker,  Davis,  Uddo,  and  Ditta  (1995)  found  that  a  lack  of 
family  cohesion  predicted  the  development  of  PTSD  in  Persian  Gulf  veterans.  Vietnam  veterans 
who  discussed  their  military  experiences  demonstrated  decreased  rates  of  PTSD  (Green,  et  al., 
1990;  Koenen,  et  al.,  2003).  Veterans  who  reported  discomfort  in  disclosing  their  Vietnam 
service  experiences  to  friends  or  family  demonstrated  an  increased  risk  for  developing  PTSD. 
Combat  exposure  increased  likelihood  of  PTSD  while  Unit  support,  or  group  cohesion,  reduced 
risk  (Roberto  &  Koob,  2009). 

Perceived  negative  community  attitudes  at  homecoming  and  community  involvement  at  timel 
were  protective  and  associated  with  decreased  risk  at  time2  (Koenen,  et  al.,  2003).  Discomfort 
in  disclosing  Vietnam  experiences  was  associated  with  an  increased  risk  for  developing  PTSD, 
but  did  not  predict  its  course  over  time.  Recovery  from  PTSD  was  significantly  influenced  by 
perceived  social  support  at  home.  Additionally,  low  morale  and  social  support  within  the  unit 
and  non-receipt  of  a  homecoming  briefing  were  associated  with  greater  risk  of  PTSD.  These 
results  raise  the  possibility  that  there  are  important  modifiable  occupational  factors  such  as 
unit  morale,  leadership,  preparing  combatants  for  their  role  in  theatre  which  may  influence  an 
individual's  risk  of  PTSD. 

The  review  by  Keane  (2006)  also  found  social  support,  intimate  partner  relationship,  and 
societal  attitude  toward  the  war  to  be  associated  with  the  development  of  PTSD. 
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Individual-level  factors 

In  a  study  of  Vietnam  veterans,  results  showed  lower  PTSD  is  associated  with  higher  education, 
older  age  at  Vietnam  entry,  higher  SES,  more  positive  parental  relationship,  more  social  support 
at  homecoming,  and  more  current  social  support  (Schnurr,  et  al.,  2003).  In  the  later  wars, 
women  have  consistently  been  found  to  experience  more  severe  PTSD  symptoms  compared  to 
men  (Breslau,  Davis,  Peterson,  &  Schultz,  2000;  Kessler,  Sonnega,  Bromet,  Hughes,  &  Nelson, 
1995).  Additionally,  younger  age  (e.g.,  Kessler,  et  al.,  1995),  minority  race/ethnicity  (e.g.,  Kulka, 
Research  Triangle,  &  United  States.  Veterans,  1988;  Ursano,  et  al.,  1999),  and  less  education 
(e.g.,  Shalev,  Peri,  Canetti,  &  Schreiber,  1996))  have  all  been  found  to  be  risk  factors  for  PTSD. 
Men,  whites,  those  reporting  more  education,  and  those  reporting  less  combat  exposure  had  a 
significantly  lower  probability  of  being  classified  into  the  PTSD  symptomatic  group  (Orcutt, 
Erickson,  &  Wolfe,  2004). 

A  recent  review  of  risk  factors  found  gender,  education,  SES  status,  history  of  child  abuse, 
intensity  and  duration  of  combat,  the  extent  of  physical  injury,  younger  age,  minority 
race/ethnicity,  history  of  psychiatric  illness,  family  history  of  anxiety,  antisocial  personality 
disorder,  history  of  childhood  adversity,  sense  of  lack  of  personal  control,  feelings  of  security, 
and  alienation  from  others,  parental  PTSD,  and  family  background  as  possible  risk  factors  for 
PTSD  (Keane,  etal.,  2006). 
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APPENDIX  H:  PRESENTATION  OF  KEY  FINDINGS  FROM  VANDERBILT 
EVALUATION:  ARMY  PDHRA  CONFERENCE 
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Presentation  Goals:  To  Describe 

•  Evaluation  Purpose  and  Design 

•  Key  Findings 

-  Effects  of  the  PDHRA 

-  Factors  Associated  with  SM  Self-reported  Problems  on 
PDHRA 

-  SM  Disclosure  on  PDHRA 

-  Unit  Leadership  and  Education  Important  to  PDHRA 
Process 

-  Clinician  and  Context  Factors 

•  Recommendations 


Evaluation  Purpose 

Assess  how  the  PDHA  and  PDHRA 
contribute  to  increasing  Service  Member 
(SM)  access  to  appropriate  care 
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Evaluation  Purpose:  Major  Tasks 

1.  Identify  factors  that  influence  the  PDHRA  process  and  SM 
satisfaction. 

2.  Describe  how  key  individuals  involved  perceive  and  carry 
out  their  roles  related  to  the  PDHRA  process. 

3.  Determine  how  the  context  of  the  clinician  screening 
influences  efficacy.  Are  phone-based  assessments 
equivalent  to  in-person  assessments? 

4.  Describe  communication  patterns  and  topics  discussed 
between  PDHRA  clinicians  and  SMs  as  it  influences  SM 
disclosure  of  problems  and  acceptance  of  referrals. 

5.  Provide  overall  evaluation  and  recommendations  = 


Evaluation  Design 

Multiple  methods  and  data  sources 
Major  components 

-  Analysis  of  PDHA  (300k),  PDHRA  (n=251  k),  and  health  care 
utilization  (n=2.1  M)  for  SMs  between  01 -JAN-06  and  15-MAR-09 

-  SM  survey  (n=6,714)  to  identify  SM-related  factors  that  may 
influence  the  PDHRA  process  and  satisfaction  with  the  process 

-  Quasi-experimental  study  (n=766)  to  examine  the  effects  of 
Battlemind  II  on  SM  reporting  and  referral  acceptance 

-  Semi-structured  interviews  with  100  key  participants  in  the 
PDHRA  process  and  observations  of  PDHRA  process  at  10 
installations 

-  Content  analysis  of  audio  recordings  of  272  PDHRA  clinician 
interview  calls  (Reserve  component  only) 

Analysis  of  PDHRA  context  (e.g.,  in-person  v.  telephone 
interview)  using  52,556  records  provided  by  contracting  agency 
for  PDHRAs  completed  between  February  2008  and  March  2009 
(Reserve  component  only)  6 
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Focus  on  Process  and  PDHRA 
Components 

•  SM  Characteristics  Accounted  for  in 
Analysis 

-  Cohort  (year  departed  theater) 

-Time  between  departure  and  PDHRA 

-  Service  Branch  and  component 

-  Combat  exposure  (from  PDHA) 

-  Deployment  location 

-  SM  total  self-reported  problems 


Generalizability  of  Findings 

•  Results  based  on  analysis  of  the  full  sample  of 
PDHA  and  PDHRA  records  can  be  generalized 

•  Results  including  analysis  of  health  care 
encounters  (HCE)  can  be  generalized  to  Active 
duty  only 

•  Results  based  on  sub-samples  (SM  survey, 
Battlemind  II  study,  semi-structured  interviews, 
audio  recordings,  and  context)  should  not  be 
generalized 
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Key  Findings: 
Effects  of  the  PDHRA 
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PDHRA  Process  Works — SMs  in 
Greater  Need  get  More  Referrals 
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Self-Reported  Problem  Count 


Number  of  SM 
problem  areas 
explains  a  substantial 
20%  of  the  variance  in 
number  of  medical 
referrals  (R2=0.20) 
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SMs  in  Greater  Need  get  More 
Referrals 

•  Those  who  got  a  referral  had  an  average  of  4.5 
problem  areas,  while  those  receiving  no  medical 
referral  endorsed  an  average  of  2.8  problem  areas. 

•  Being  under  care  did  not  influence  whether  or  not 
SMs  received  a  medical  referral 

•  No  evidence  that  SMs  declining  a  referral  is  related  to 

-  Previous  referral  from  PDHA 

-  Health  care  encounters  between  PDHA  and  PDHRA 


Active  duty  SMs  With  Medical  Referrals 
Have  More  Health  Care  Encounters  (HCE) 

•  All  SMs  had  more  care  after  PDHRA 

-  HCE  increased  50%  after  the  PDHRA  regardless  of 
referral 

-  From  0.9  HCE  (in  6  weeks  prior)  to  1.5  HCEs  (in  6 
weeks  post)  on  average 

•  Compared  to  6  weeks  prior  SMs  with  medical 
referrals  had  more  HCEs  in  6  weeks  post-PDHRA 

-  0.5  more  HCEs  on  average  with  no  medical  referral 

-  1.2  more  HCEs  on  average  with  medical  referral 
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SMs  Generally  Satisfied  with  the 
PDHRA  Process 

•  SMs  were  neutral  to  positive  that  the 
PDHRA  helped  them  to  identify  their 
concerns  (3.3  on  a  1-5  scale) 

•  SM  satisfaction  with  the  PDHRA  clinician 
averaged  3.6  (on  a  1-5  scale) 


Unit  Leaders  are  Ambivalent 
About  the  PDHRA 

•  Many  unit  leaders  indicated  that  the  PDHRA 
positively  affected  their  SMs’  military  readiness 

•  However,  unit  leaders  rarely  receive  feedback 
so  no  way  to  measure  PDHRA  effectiveness 

•  Majority  did  not  feel  that  the  process  helped 
them  to  identify  SMs  with  physical  or  mental 
health  problems 

•  Unit  leaders  had  concerns  about 

-  Over- identifying  SMs  as  having  problems 

-  Time  away  from  other  duties  and  conflicting  priorities 
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Key  Findings: 

Factors  Associated  with  SM  Self- 
Reported  Problems  on  PDHRA 


PDHA  Responses  Strongly  Predict  PDHRA 

•  Symptoms  reported  at  the  PDHA  were  also  likely 
to  be  endorsed  at  the  PDHRA  (correlations 
between  0.31  and  0.54) 

-  80%  of  SMs  who  reported  a  problem  on  PDHA  also  reported  a  problem  on 
PDHRA 

-  Particularly  true  for  TB I  (OR=21.6),  PTSD  (OR=7.7),  and  depressive 
symptoms  (OR=8.0) 

-  We  did  not  examine  the  percentage  of  new  symptoms  reported  on  the 
PDHRA  when  not  previously  reported  on  the  PDHA 

•  Intervening  HCEs  did  not  influence  PDHRA 
reporting 

•  Supports  usefulness  of  the  PDHA  in  predicting 
specific  problems  on  PDHRA 

-  Considering  only  the  self-report,  the  best  predictor  of  endorsing  a 
symptom  on  the  PDHRA  is  having  previously  endorsed  the  same 
symptom  on  the  PDHA.  Having  endorsed  any  other  problem  on  the  PDHA 
is  a  much  weaker  predictor  of  a  specific  problem  on  the  PDHRA. 
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Combat  Exposure  Leads  to  Greater 
Likelihood  of  SM  Problems  on 
PDHA/RA 

•  SMs  who  reported  combat  exposure  on  PDHA 
were 

-  Almost  three  times  more  likely  to  report  any  problem  on 
the  PDHA 

-  Twice  as  likely  to  endorse  any  problem  area  on  the 
PDHRA 

•  Significant  but  small  relationship  to  the  number  of 
self-reported  problem  areas 

-  PDHA  (R2=0.10) 

-  PDHRA  (R2=0.12) 


Many  SMs  Report  Behavioral  Health 
Symptoms  on  the  PDHRA 

•  Behavioral  health  problem  areas  reported  by  SMs 

-  24%  one  or  more  symptoms  of  PTSD 

-  10%  one  or  more  symptoms  of  depression 

-  13%  single  item  on  relationship  conflict 

•  These  rates  are  similar  to  SM  reports  of  other 
types  of  problems 

-  29%  one  or  more  physical  health  symptoms 

-  25%  one  or  more  exposure  concerns 

-  14%  one  or  more  TBI  symptoms 

18 
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Key  Findings: 

SM  Disclosure  on  PDHRA 


19 


SM  Survey  Methods 

•  Vanderbilt  developed  a  survey  to  identify  SM- 
related  factors  that  may  influence  the  PDHRA 
process  and  satisfaction  with  the  process 

•  All  surveys  were  anonymous 

•  Overall  6,714  surveys  collected  from  SMs  who 
just  completed  PDHRA 

-  3,768  collected  by  LHI  at  34  travelling  team  events 

-  2,936  collected  by  VU  at  9  site  visits,  and  some  Air 
Force  SMs  (n=256)  completed  the  survey  online. 

-  2,217  surveys  were  linked  to  PDHRAs  using  methods 
that  did  not  compromise  anonymity 
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Many  SMs  Admit  to  Underreporting 
Problems  on  PDHRA 

•  Asked  anonymously,  two-thirds  of  SMs  agreed 
that  they  had  fully  disclosed  physical,  emotional, 
or  alcohol  use  concerns 

•  1 0-14%  of  SMs  reported  that  they  did  not  fully 
disclose  these  problems 

•  An  additional  25%  chose  not  to  indicate  (neither 
agree  nor  disagree)  the  status  of  their  disclosure 

•  Clinicians  interviewed  about  PDHRA  process 
estimated  about  one-third  of  SMs  do  not  fully 
disclose  on  PDHRA 


SMs  Underreport  Behavioral  Problems  on 
PDHRA  Compared  to  Anonymous  Survey 

•  On  the  SM  survey,  39%  of  SMs  either  reported  a  problem 
and/or  had  friends  or  family  members  suggest  they  seek 
help 

-  34%  of  SMs  reported  experiencing  an  emotional,  alcohol,  stress, 
or  family  problem  since  deployment 

-  22%  had  friends  or  family  suggest  seeking  help  for  such  a  problem 

•  These  SMs  had  more  negative  attitudes  toward  help¬ 
seeking  and  accepting  mental  health  treatment  than  SMs 
who  did  not  report  a  problem  on  the  SM  survey 

•  43%  did  not  report  any  behavioral  health 
problems  on  PDHRA 

22 
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Concerns  about  Confidentiality  may 
Discourage  SM  Disclosure  on  PDHRA 

•  SMs  were  less  likely  to  agree  that  they  had  fully 
disclosed  on  the  PDHRA  when  they 

-  Were  seeking  promotion  in  the  next  6  months 

-  Knew  the  PDHRA  clinician  before  the  interview 

•  Only  25%  knew  that  current  DoD  policy  no  longer 
requires  disclosing  deployment-related  mental 
health  treatment  on  Q21  of  SF86  when  applying 
for  security  clearance 

•  Inadequate  privacy  for  clinician  interview 
observed  during  2  of  10  site  visits 


Stigma  May  not  be  Directly  Related  to 
Disclosure  on  the  PDHRA 

•  No  evidence  of  a  direct  relationship  between  stigma  and 
our  measures  of  PDHRA  self  disclosure  in  this  evaluation 

•  There  may  be  complex  indirect  relationships  requiring 
further  analysis 

-  Stigma  related  to  disclosure  greater  for  SMs: 

•  Seeking  promotion  in  next  6  months 

•  Experiencing  an  emotional,  alcohol,  family,  stress 
problem  since  deployment 

•  Friends/family  suggested  seeking  help  for  such  a 
problem 

-  These  3  variables  associated  with  SMs  indicating  less 
than  full  disclosure  on  PDHRA 

24 
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Informal  Support  from  Family  and 
Friends  may  Encourage  Disclosure  on 

the  PDHRA 

•  Among  SMswho  reported  emotional,  alcohol, 
stress,  or  family  problems  on  the  SM  survey 

-  Few  (30%)  sought  help  from  medical/mental  health 
professionals 

-  Even  fewer  (22%)  talked  to  religious/spiritual  leaders 

-  A  majority  (74%)  had  spoken  to  family  or  friends 

•  SMs  who  had  spoken  to  family/friends  reported 
greater  post-deployment  support  and  indicated 
they  were  more  willing  to  fully  disclose  on  the 
PDHRA 


PDHRA  Education  May  Encourage 
SM  Disclosure 

•  SMs  were  more  likely  to  agree  that  they  had 
fully  disclosed  on  the  PDHRA  when 

-  Briefed  on  the  PDHRA  from  a  unit  leader 

-  Received  education  on  post-deployment  and 
reintegration  issues  in  the  form  of  written 
materials,  film/video,  or  websites 

-Exposed  to  Battlemind  II  (7-10%  higher  than 
SMs  not  exposed) 
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Yet  Education  not  Widely  Used  nor 
Consistently  Available 

•  Pre-briefs  varied  as  observed  during  PDHRA  events 

-  6/10  events  where  PDHRA  mentioned  by  name 

-  3/10  events  where  PDHRA  process  explicitly 
explained 

•  About  half  or  fewer  SMs  reported  using  deployment 
cycle  educational  materials  prior  to  completing  PDHRA 

•  Yet  when  these  materials  were  used,  the  majority  of 
SMs  found  them  helpful 

27 


Key  Findings: 

Unit  Leadership  and  Education 
Important  to  PDHRA  Process 
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Unit  Leader  Involvement  and  Support 
Important  to  PDHRA  Process 

•  Command  support  important  in  educating  SMs  and 
generally  setting  the  stage  to  encourage  SM  openness 
during  the  process 

•  More  positive  SM  attitudes  about  post-deployment  support 
and  help  seeking,  PDHRA  leadership  support,  and  unit 
cohesion  for  personal  problems  when: 

-  At  least  one  NCO  or  Officer  in  theater  with  them 

-  Unit  NCO  or  Officer  briefed  them  on  the  PDHRA 

•  PDHRA  leadership  support  related  to  more  positive  SM 
attitudes 

-  Higher  post-deployment  support  and  help  seeking,  unit  cohesion 
for  personal  problems,  satisfaction  with  the  PDHRA  clinician, 
general  willingness  to  self-disclose 

-  Lower  perceived  stigma  related  to  disclosure  and  fewer  barriers  to 

accepting  mental  health  referral  29 


Quasi-experimental  Study  of 
Battlemind  II 

•  Design 

-  Two  groups 

•  Group  1 :  SMs  exposed  to  Battlemind  II  video  and  discussion 
prior  to  PDHRA 

•  Group  2:  SMs  did  not  receive  Battlemind  II  training  prior  to 
PDHRA 

-  Originally  intended  as  randomized  control  study; 
however  units  assigned  to  groups  based  on  schedule 
(PDHRA  event  on  different  days) 

•  Participants 

-  One  site  included  in  analyses  (Army  Active) 

-  501  SMs  in  BMII  condition,  265  SMs  in  No  BMII 
condition 

30 
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Battlemind  II  related  to  more  positive 
attitudes  to  PDHRA  on  SM  Survey 

•  Similar  rates  of  anonymous  reporting  of 
behavioral  health  problems  regardless  of 
Battlemind  II  exposure 

•  SMs  exposed  to  Battlemind  II 

•  Had  more  positive  attitudes  toward  the  PDHRA 

•  More  post-deployment  support  and  help  seeking 

•  Higher  satisfaction  with  the  PDHRA  clinician 

•  More  general  willingness  to  self-disclose 

•  Less  perceived  stigma  related  to  disclosure 

•  Fewer  barriers  to  accepting  mental  health  referral 

•  Agreed  that  they  more  fully  disclosed  on  the  PDHRA 

•  Physical  health  concerns  (66%  v  55%) 

•  Emotional  health  concerns  (62%  v  50%) 

•  Alcohol  use  health  concerns  (65%  v  52%) 


BMII  Study  Conclusions 

•  Psychosocial  education  such  as  Battlemind 
II  may  improve  the  PDHRA  process 

•  Study  is  limited  by  its  focus  on  attitudes 
only 

•  Additional  research  needed  to  understand  if 
BMII  leads  to  actual  changes  in  disclosure 
on  PDHRA,  which  can  be  found  in  our  data 
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Key  Findings: 

Clinician  and  Context  Factors 


33 


Clinical  Interview  Focuses  on  SM 
Self-Reported  Problems 

•  According  to  on-site  clinicians  interviewed  by  VU  staff 

-  Primary  purpose  of  the  PDHRA  was  to  identify  and 
address  SM  concerns  and  get  them  the  help  they  need 

-  Clinicians  reported  using  positive  responses  from  the 
SM  self-report  and  built-in  alerts  (e.g.,  for  alcohol  use 
problems)  to  guide  the  interview  in  addition  to  SM  eye 
contact,  sincerity,  and  expressions 

•  Results  from  secondary  analysis,  SM  survey  and  analysis 
of  de-identified  audio  recordings  of  PDHRA  interviews  are 
consistent  with  findings  from  interviews 

•  Interview  provides  value  in  confirming  problems,  but  does 

little  to  increase  sensitivity  of  the  clinical  interview 
(i.e.,  increasing  disclosure  beyond  what  is  already  in 
SM  self-report) 
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Clinician  Interview  Adds  Value  to 
Increasing  Medical  Referrals 

•  The  number  of  SM-reported  problem  areas 
are  the  main  predictor  of  a  medical  referral 
(R2=0.20) 

•  After  accounting  for  SM  problems  and 
characteristics,  the  clinical  interview  as 
documented  on  the  PDHRA  adds  a 
relatively  small  but  significant  contribution 
(R2=0.07) 


Number  of  Major  Clinician  Concerns 
Increases  Referrals 


Probability  of  medical  referral  at 
differing  levels  of  clinician  concerns 

When  SMs  endorsed 
5  problem  areas 

•  No  major  clinician 
concerns  ->  35% 
probability  of  medical 
referral 

•  5  major  clinician  concerns 
->  55%  probability  of 
medical  referral 

36 


-No  Major  Clinician  Concern 
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Clinical  Interview  Misses  Many  Problems 
for  SMs  Who  Do  Not  Disclose  them  on 
PDHRA  Self  Report 

•  We  compared  (a)  number  of  clinician  major  concerns  and  (b)  whether 
a  medical  referral  was  given  for  two  groups  of  SMs: 

•  Non-  Disclosers  -  reported  behavioral  health  problems  on  SM 
survey,  but  not  on  PDHRA  (n=282) 

•  Disclosers  -  reported  behavioral  health  problems  on  SM  survey 
and  on  PDHRA  (n=380) 

•  Findings 

-  (a)  Average  number  of  clinician  major  concerns  was  five  times 
lower  for  SMs  who  did  not  disclose  on  the  PDHRA  (means  0.11 
vs.  0.53) 

-  (b)  Receipt  of  one  or  more  medical  referrals  (for  any  reason) 
was  almost  three  times  lower  for  SMs  who  did  not  disclose  on 
the  PDHRA  (means  0.14  vs.  0.67) 
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Low  Reliability  in  Clinician  Interviews 
Across  Multiple  PDHRAs 

•  Multiple  completions  for  a  single  deployment  were  compared  (3,720 
records;  1860  SMs) 

•  More  stable  over  time  for  SM  self-report  compared  to  clinical  portion 

-  SM  self-report  correlations  very  strong  (r=0.6-0.9) 

-  Clinician  portion  correlations  were  much  lower  (r=0.2-0.5) 

•  Poor  reliability  of  clinician  interview  not  likely  due  to  intervening  HCE 
or  changes  in  SM  health  status 

-  SM  self-reports  within  one  week  (aggregated  r=0.88) 

-  Clinician  portion  within  one  week 

•  Risk  assessment  (r=0. 55) 

•  Major  concerns  (r=0.43) 

•  Referrals  (r=0.38) 

•  Note  that  clinicians  are  most  likely  different  people  at  different  times 

•  The  lack  of  consistency  in  the  clinician  interview  may  be  due  to 
differences  in  clinician  interview  approach  and  documentation 

38 


133 


Annual  Report:  Contract  #  W81XWH-09-2-0172 


There  is  a  Lack  of  Systematic  and 

Intensive  Training  Specific  to  PDHRA 

•  Clinicians  interviewed  about  PDHRA 
process  indicated  that  training  was 
generally  limited  to  shadowing  of  other 
clinicians 

•  Structured  feedback  not  routinely  provided 

•  Potential  reason  for  the  low  reliability 


Lack  of  Time  and  High  Caseloads  for 
Interview  May  Limit  Effectiveness 

•  PDHRA  clinician  interviews  generally  short 

-  59%  of  SMs  surveyed  reported  duration  of  10  minutes 
or  less 

-  Interview  duration  observed  during  site  visits  were  less 
than  10  minutes  on  average  at  most  sites 

•  Interviews  indicated  limited  availability  of  sufficient  number 
of  clinicians 

•  Clinicians  concerned  about  being  able  to  be  effective  in 
the  time  allotted  to  process  each  SM 

-  Particularly  with  regard  to  establishing  a  meaningful 
rapport 

-  SMs  sometimes  rushed  through  interview  due  to 
substantial  time  already  spent  waiting 

40 
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Coding  Communication  Patterns  in 
Audiotaped  Clinician  Interviews 

•  PDHRA  specific  coding 

-  Problem  area  mentioned? 

-  Education  provided? 

-  Previous  treatment  queried? 

•  The  Roter  Interaction  Analysis  System  (RIAS)  is  an  internationally 
recognized  instrument  that  has  emerged  as  the  most  widely  used 
system  for  coding  communication  in  medical  encounters 

•  Numerous  sub-codes  can  be  used  to  identify  the  nature  of  information 
given  and  received  as  well  as  the  general  tone  of  the  medical 
encounter 

•  RIAS  codes  for 

-  Task-focused  exchanges:  have  to  do  with  the  interview  process  (e.g., 
asking  questions  about  previous  treatment  or  giving  medical  information) 

-  Socio-emotional  exchanges:  more  personal  (e.g.,  laughing  or  making  an 
empathic  statement) 

41 


PDHRA-specific  communication  about 
behavioral  health  issues  different  than  physical 

health 

•  Less  mention  of  behavioral  health  problem  areas 

-  Physical  health  mentioned  regardless  of  SM  endorsement  (87%  v 
84%) 

-  Behavioral  topics  mentioned  more  when  SM  endorsed  (64%)  than 
not  endorsed  (35%) 

•  Little  provision  of  education 

-  Education  related  to  mental  health  issues  in  14%  of  all  calls 

-  Increased  to  24%  in  calls  where  a  medical  referral  was  given 

•  Prior  treatment  discussed  less  for  behavioral  health  and 
exposure  problems 

-  Physical  health  or  TBI  problems  (87%  to  92%) 

-  Alcohol,  mental  health,  or  exposure  (3.1%  to  63.6%) 

•  We  do  not  know  what  these  patterns  look  like  for  in-person 
interviews 

42 
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Clinicians  do  not  use  Empathy 
Statements 

•  ‘Clinician  empathy  statements’  (e.g.,  “This  is  distressing 
for  you,  I  understand”)  occurred  in  only  nine  (3.3  %)  calls 

•  The  majority  of  RIAS  variables,  18  of  35,  occurred  in  less 
than  50%  of  calls.  This  illustrates  that  there  was  a  great 
degree  of  variation  in  the  types  of  communication 
clinicians  used 

•  Further,  variability  in  clinicians’  communication  patterns  is 
mostly  related  to  SM  self-report  characteristics 
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Comparison  of  PDHRA  Results  by 
Context:  Telephone  v.  in-person 
Interviews 

•  Reserve  component  only 

•  A  controlled  study  with  random  assignment  would  have 
been  ideal,  but  no  volunteer  units  could  be  recruited 

•  Analyses  were  conducted  after  creating  equivalent  groups 
of  SMs  using  statistical  methods  due  to  differences  in  SM 
characteristics  and  PDHRA  self-report 
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Interview  Context  Matters 

•  Documentation  of  major  clinician  concerns  was 
similar 

•  Clinician  risk  assessment  for  presence  of  TBI  and  alcohol 
problems 

-  More  likely  to  be  documented  as  a  potential  problem  during 
telephone  interviews 

-  Less  likely  to  indicate  referral  during  telephone  interviews 

-  Stronger  positive  correlation  between  risk  assessment  and 
indication  of  referral  for  in-person  interviews 

•  Medical  referrals  more  likely  for  PDHRAs  with  in-person 
interviews 

-  Three  times  more  likely  to  be  given 

-  SMs  half  as  likely  to  decline  any  referral 

•  It  is  not  clear  that  more  referrals  are  better 

-  Call  center  clinicians  do  many  more  screenings  than  in-person 
clinicians,  and  may  be  appropriately  increasing  specificity  45 


Screening  for  Alcohol  Use  may  be 
Creating  Too  Many  False  Positives 

•  Additional  questions  and  algorithm  added  to  PDHRA  2008 
version 

-  Greatly  increased  number  of  positive  screenings  compared  to 
previous  versions  (12%  to  42%) 

-  However,  major  clinician  concerns  and  referrals  for  alcohol  did  not 
increase  between  versions 

•  In  audio-recordings  of  PDHRA  interviews,  some  clinicians 
pose  alcohol  questions  differently 

-  Downplay  symptoms  “So  do  you  have  an  alcohol  problem  or  is  this 
just  social  drinking?” 

-  Discomfort  with  scoring  system  “The  military  has  a  scoring  system- 
a  very  harsh  scoring  system  for  how  much  they  feel  people  should 
drink* 

•  Some  behavioral  health  consultants  interviewed  by  VU 
staff  expressed  concern  that  algorithm  too  sensitive 
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Recommendations 


47 


Recommendations: 
Implementable  Now 

•  Establish  quality  assurance  procedures  for  the  clinician 
interview 

•  Establish  clinician  guidelines  for  a  more  structured  and 
systematic  PDHRA  interview 

•  Encourage  unit  leadership  involvement  in  and  support  of 
the  PDHRA  process 

•  Provide  greater  visibility  and  incentive  for  SMs  to  take 
advantage  of  education  for  post-deployment  and 
reintegration  issues 

•  Offer  PDHRA-specific  education  targeted  to  informal 
supports  (religious  and  spiritual  leaders  and  SMs’  families 
and  friends)  to  increase  awareness  of  the  PDHRA  as  a 
helpful  source  of  support 

•  Ensure  greater  confidentiality  during  the  PDHRA  process 

•  Add  questions  to  the  PDHRA  regarding  SMs'  combat  48 
exposure  (or  make  PDHA  available) 
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Actions  that  Have  Already  Been 
Taken  by  LHI 

•  In  June  2009,  revised  the  healthcare  provider  (HCP)  QA  review  form 
to  better  check  if  HCPs  are 

-  Providing  education 

-  Asking  questions  about  all  types  of  potential  problems  (including 
exposure  and  behavioral  health) 

•  Adding  questions  on  the  HCP  QA  review  form  to  check  if  HCPs  are: 

-  Asking  the  behavioral  risk  questions  verbatim 

-  Asking  about  treatment  for  alcohol  and  behavioral  health  concerns 

•  Adding  the  following  to  the  training  of  HCPs  (both  Call  Center  and 
traveling  teams) 

-  All  of  above 

-  How  to  deal  with  the  alcohol  questions,  especially  for  those  with 
AUDIT  scores  at  or  slightly  above  the  cut-off  for  potential  problems 
so  the  HCPs  don’t  alienate  the  Service  member,  yet  don’t  minimize 
the  potential  problem 

-  Increased  normalization  of  behavioral  health  issues  upon  return 
from  deployment 

•  Working  with  Call  Center  HCPs  to  reduce  threshold  for  referrals  and 

decrease  declinations  49 


Recommendations:  Implementation 
Requires  Further  Development 

•  Require  all  clinicians  to  have  successfully 
completed  PDHRA-specific  training 

•  Provide  clinicians  with  monitoring  and 
feedback  about  their  performance  on  the 
PDHRA 

•  Re-evaluate  the  PDHRA  alcohol  screening 
and  training  provided  the  clinicians  on  this 
topic 
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Next  Steps  for  Vanderbilt 

•  Two-year  cooperative  agreement  with  USAMRMC 
to  develop  and  pilot  PDHRA-specific  clinician 
training 

•  Intend  to  use  these  results  to  inform  activities  with 
expert  panel  input 

-  Identify  and  evaluate  innovative  PDHRA 
training  programs  for  clinicians 

-  Develop  and  implement  a  training  and 
feedback  system  for  PDHRA  clinicians 


Questions  Raised  by  Evaluation 
-  Data  Available 

•  Expand  analysis  of  the  SM  survey  to  explore  the 
relationship  between  the  attitudes  from  survey 
and  PDHRA  SM  self-report,  clinician  concerns 
and  referrals 

•  Examine  ICD-9  and  CPT  codes  to  see  if  HCEs 
after  the  PDHRA  relate  to  the  types  of  concerns 
and  referrals  documented  on  the  PDHRA.  This 
analysis  is  one  way  to  understand  if  referrals  from 
the  PDHRA  are  appropriate 

•  A  more  comprehensive  analysis  could  examine 
additional  differences  between  the  2005  and  2008 
versions  of  the  PDHRA 
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Additional  Data  Collection  Required 

A  randomized  study  to  examine  if  context  of  the  clinical 
interview  influences  SM  disclosure  of  symptoms  or 
acceptance  of  referrals.  Are  face-to-face  interviews 
worth  the  cost? 

Conduct  a  randomized  experiment  with  SMs  randomly 
assigned  to  one  of  the  following  groups: 

Clinician  assessment  only  (the  SM  sees  the  clinician,  but  does 
not  complete  a  PDHRA  form) 

-  Blind  to  self-report  (the  SM  completes  the  self-report  but  the 
clinician  does  not  see  it) 

Self-report  and  clinician  assessment  (the  typical  PDHRA 
process) 

-  Control  (PDHRA  delayed  by  at  least  two  months) 

-  Comparison  of  clinician  concerns  and  referrals,  and  SM  health 
care  encounters  among  these  groups  would  help  establish  the 
role  of  the  clinician  in  the  PDHRA  process. 

Determine  if  the  services  to  which  SMs  are  referred 
improve  outcomes 

53 
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